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Sfie subjects were 4b jantor enliisted Army personnel scheduled to take the 
Russian Basic Course (^®C) i They were randomly selected and sorted into two 
sections each for the suggestopedia and standard groups. One section of 10 
junior enlisted Army and Navy personnel comprised the flexibly scheduled group. 

Analyses of variance^ combined with subsequent cne-tailed t-tests, found 
^^^^ the suggeistdpedia group had significantly lower scores than the two 
P^??^?_8???P?_??_ the written and oral cdmpbhents of achievement tests . Similar 
^■^??M*^^?^_^^???^™9f^_™9^8 S^oupis were found bh the reading and speaking 

components of the Proficiency Advancement Test (PAT), No significant differ- 
ences were found among groups on the listening component of the PAT. There 
were no significant differences between the standard and flexibly scheduled 
groups oh the achievement tests, the PAT, or on the face-to-face oral 
interviews. 



?^?P^?^_^°5?_?°^9?8_S?^^PS_ pn^^ ^ various attitudihal measures indicated. 

signif ic^tly more positive attitudes by the suggestopedia group tbward their 
instructor for the first 4 weeks when compared to the standard group. TheriB 
were ho differences between groups on any other results from the weekly 
surveys, and between pre- and post tests results for any of the groups i On 
the 12 individual scales comprising the end-of-course questionnaire, results 
.similar among grbups bh 9 of the scales. Onthe "Language. Use Anxiety" 
scale, the suggestopedia group indicated mbre cbmfort with the language; ph 
the "Effort Required" scale, they felt that the methbdblogy required little br 
no effort to learn the language; and on the "Course Materials" scale, the twb 
DLIFLC groups felt more positive about their course materials. 

In conclusion^ suggestopedia ne.^.ther accelerated learning nor resulted in: 
inbre bverall positive attitudes in students when compared to either the 
standard or flexibly scheduled groups. The evaluation of the flexible- 
scheduling methodology did indicate a time saving of approximately 1 week. 
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FOREWORD 



The Presidio of Monterey Field Unit of the U.S. Army Research Institute 

has as Its primary mission the execution of research to Improve training to 

better meet unit mission requirements. One aispect of such training has been 
the area of foreign language. 

"^^^ Presidential eommission Report (1979) and the 1982 Government Account- 
ing Office (GAO) report entitled "Weaknesise in the Resident Language Training 
System of Defense Language Institute Affect the Quality of Trained Linguists" 
emphasized the need to identify alternative methodologies for teaching languages. 

The Defense Language Institute Foreign Language Center (DLIFLG) currently 
uses the Progressive Skill Integratidh (PSI) approach, a functional skill- 
building approach to language learning that progresses through a number of 
stages, beginning with the perception of new concepts and culminating with the 
acquisition of working communication skills. Though this system has been 
effective, the DLIFLC and other Army language trainers continue to examine 
methods to improve training. 

This report provides a comprehensive summary of a research effort that 

compared and evaluated three of these methods: the suggestopedia, a flexible- 
echeduiing methodology, and the standard DLIFLC methodology currently used. 




EDGAR M._ JOHNSON 
Technical Director 
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S.CpMPARISON OF INNOVATIVE TRAINING TECHNIQUES AT THE DEFENSE LANGUAGE INSTITOTE 
FOREIGN LANGUAGE CENTER 



EXECUTIVE SUMMARY 



Requirement: 



The Defense Language Institute Foreign Language Center (DLIFLC) and the 
Soldxer Support Center (SSC) requested that ths Aimy Research Institute (ARI) 
assess the effectiveness of and students ' attitudes toward suggestopedia 
instruction relative to the standard DtlFLC instruction currently employed. 
The study also included a comparison of a flexible-scheduling methodology 
identxfxed by the DLIFLC as a target of opportunity. 



Procedure : 



Fxfty junior enlisted Army and Navy personnel were initially examined. 
Twenty Army students received the suggestopedia instruction, and 20 received 
the standard DLIFLC instruction. Ten Army and Navy students received the 
flexible-scheduling instructional metiaodology. 

Achievement tests, a Proficiency Advancement Test (PAT), and face-t5-face 
oral interviews were used as criteria of training ef f ectivness. Achievement 
tests were administered after tiie presentation of a block of instruction caile^^ 
a module. The PAT and oral interviews were administered to each group when 
they completed the course materials comprising Term I, the period used for this 
study. Completion dates were 10 weeks for the suggestopedia methodology, 14 
weeks as retrospectively observed for the flexible-scheduling methodology, and 
15 weeks as normally planned for the standard DLIFLC methodology. Attitudes 
were assessed at the beginning and end of the term, and at weekly intervals, 
student demographic variables were represented by military rank, military 
occupational specialty (MOS), age, years of military service, educational 
level, prior language training, and gender. Additional descriptive variables 
for the student population included Defense Language Aptitude Battery (DLAB) 
scores. General Technical (GT) scores, and a pretest of attitudes about the 
potential advantages of learning Russian and about foreign languages in general. 

Findings: 

The available data showed that the suggestopedia methodology was not as 
effective after 10 weeks of study as tha flexibly scheduled instruction af^er 
14 weeks or the standard DLIFLC instruction after 15 weeks. In fact, the 
suggestopedia group had significantly lower scores On all measures of academic 
performance with the exception of results on the listening component of the PAT 
and the oral interviews. Measures of attitudes indicated that tl^e suggesi-.opedia 
group was more positive toward their instructor (s ) during lAe first 4 weeks 



EXECUIIgE -SUMMARY (Continued j 



but then fell to the level of the two DLIFLC groups. On attitudes compared at 
the conclusion of the course, the suggestopedla group generally felt more 
comfortable about Its ability to use the Russian language and felt that less 
effort had been required when compared to the other groups. The flexibly 
scheduled and standard DLIFLC groups felt more positive about their course 
materials, but also had more hands-on (workbooks) and take-bdtne materials 
(workbooks and tescbboks)^ A comparison between pre- and posttest results 
Indicated no attitude changes within these two groups. 

Utilization of Findings: 



This research Implies that suggestopedla, as a whole, would not justify a 
change In the istandard DLIFLC Instructional methdddldgy* However, suggestopedla 's 
use as an enrichment adjunct to the curriculum may warrant further research. 
Additionally, components of the suggestopedla methodology may affect factors 
such as student attrition and memory retention, thereby warranting further 
research. 

The evaluation of the flexible-scheduling methodology Indlcaties that 
students can progress through the curriculum at a faster pace than usual. 
Further research may be necessary to determine whether this Is an effect of the 
Instruction or the curriculum. 
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A COMPARISON.QP iNNOVATiVE TRAINING TECHNIQUES 

AT THE DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER 



INTRODUCTION 



As part of the military's continued efforts to improve the quality of f6i- 
eign language training, the Soldier Support Center (SSC) and Defuse Language 

fSff?^^S''KfLf"^"^fV^"'" ^°''^^^^^ submitted a Concept Evaluation Plan 
tCEP) to the Training and Doctrine Command (TRADOC) for the evaluation of an 
i^I^*'^''^ training methodology called saggestopedia. In September, 1984 
TRADOC approved^the CEP and a contract was awarded to the Lozanov Learning Sys- 
tems, Inc.jLLSI) to provide a resident course of instruction using the suggest- 
opedia methodology. The Army Research Institute (ARI) was asked to conduct the 
evaluation of ^suggestopedia as compared to the standard instructional methodology 
currently used at the DLIFLC. A third methodology, using flexible scheduling 
of the presentation of materials, was modified from the standard DLlFLC course 
fn. ^^^f"''^^ ' ^^^""u^^'^ as a "target of opportunity," and added to the study 

ll thriS'i"""*.. 5^°^^^^^"^^ °^ '^^^^"^"^^ '^°"'P^" the effectiveness 

of the three methodologies using measures of academic performance, and analysis 
of students' attitudes about the respective instructional methodology. 

The suggestopedia methodology is a unified system of instruction character- 
ized by a variety of techniques emphasizing a relaxed and positive learning 
atmosphere. The instruction is delivered in situational contexts maximizing 
^ oral communicative skills (proficiency). The standard DLIFLC 
methodology used a Progressive Skill Integration (PSI) approach, which is a 
functional approach to language teaching that stresses the integration of the 
various components of language (i.e. , pronunciation, grammar, vocabulary, writ- 
ing systems, etc.) into communication skills, it is a progressive approach in 
that students advance through a number of stages beginning with the perception 
'concepts and culminating with the acquisition of working communication 

§*, flexible-scheduling treatment is similar to the standard DLIFLC 

methodology except that the former uses a pacing of the presentation of mate- 
rials based upon group readiness rather than a fixed schedule for the presen- 
tation of materials. Further discussion characterizing the methodologies may 
Treatments!"" ^" ""^^^ chapter under the subtitle, "Description of Instructional 

Data from this study may be used in formulating subsequent research efforts 
on components of effective instruction that characterize the methodologies (e.g.. 
functional practice, positive reinforcement, cueing, positive role modeling), 
individual components found effective may be incorporated throughout a course 
of instruction. 



Background 

A review of the literature indicates mixed findings regarding suggestive- 
accelerated leartng or suggestopedia. In addition, there is little evidence 
reflecting the study of suggestopedia in a military setting or with military 
personnels ^ 
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One review conducted within the Federal government was a self report 
experience with National Security Agency personnel at a five-week 
suggestopedia Russian Course cited by Shitama ( 1982) ; Overall remarks made by 
the ten NSA students and two NSA instnictbrs (who only attended the one-week 
Lbzahov instructor training workshop) concluded that while some features of 
suggestopedia wci^e interesting. and helpful^ such as the use of mlisic and 
relaxation techniques, the collective methodblpgy is not recommended for 
individuals requiring a thorough grasp of the language. A report which 
involved Special Forces military personnel learning German with the 
suggestopedia methodology was conducted at Fort Devensj Massachusetts* 
Findings indicated a reduction in course length from 12 to 6 weeks. However^ 
10 of the 12 students had been previously exposed to German either in school 
or Overseas (Dhorityt 1984)* 

Studies conducted by Lozanov, the creator and developer of suggestopedia, 
suggest high success rates covering a range of topics from foreign language 
training to training in mathematics at virtually all age levels (Lozanov, 
1978) • One example is the teaching of a year of traditional foreign language 
curriculum in three and one-half months (tozanov, i575)i However, these 
findings and others indicating high success rates have been found to lack 
"scientific validity" as noted by Bancroft (1976) and Scovel (1979) in his 
review of Lozanov's suggestopedia. 

klitie (1974) found in many of Lozanov's studies highly motivated adults, 
experienced language teachers, and perfect attendance records which may result 
in high success rates regardless of the instructional methodology used, in an 
attempt to replicate some of Ldzanov's work, Kline indicated that vocabulary 
achievement was approximately half of that reported by Lozanov. 

Other studies (Behitez-Bordbti & Schuster, 1976) indicate positive results 
but only for two-hour sessions once a week aud with no control group. An 
experiment in teaching beginning Russian (Kurkdv, 1971) indicated higher 
success rates for suggestopedia over traditional instruction, but it was noted 
that approximately twice as many students in the suggestopedia group had prior 
formal Russian language training. Additionally, the students in the 
suggestopedia group indicated that they needed to spend more time than usual 
in outside preparatibn such as vbcabulkry copying and reading gratnmar. Reiiard 
(1976) reviewed a study using suggestopedia to learn French but found that the 
suggestopedia group was comprised of volunteers while the control group was 
not. 

Wagner and Tilney (1982) conducted ah experiment comparing suggestopedia 
with traditional instructional methods for learning over five weeki. Results 
indicated no significant improvement by the suggestopedia group. In fact ^ it 
was found that the traditional group learned significantly more vocabulary. 

Other studies evaluated characteristics found in the suggestopedia 
metbodologyi Several findings indicated an increase in performance levels for 
verbal learning and retention when students were provided with some relaxation 
techniques to reduce stress before testing on difficult material (Straughn & 
bufort, 1969; Ghaney & Andreasen, 1972) * However, studies by Martin and 
Schuster (±977) and Lipsitt (1963) found that there seemed to be an optimal 
level of stress for learning. Some students were found to learn better if 
they felt a certain amount of stress. Overall the findings seemed to indicate 
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a curvilinear relationship between stress and learnings Learning increased as 
stress increased up to an optimum level, after which leariling fell off as a 
function of increases in stressi This demonstrates the Yerkes-Dodson Law* 
For further reading see Sanders, Eng, and Murph (1985). 

The components of music and imagery adapted from suggestopedia were 
studied for their separate and combined impact. Stein (1982) found that the 
addition of nrasic and imagery together and music separately indicated a 
significant increase in the retention of vocabulary for college students when 
compared to the control group without treatment. Subjects in the music plus 
imagery group heard Handel's Water Music^ and the experimenter read aloud the 
words to be studied. Subjects in the music only group heard just the music. 
The control group subjects had neither the words spoken nor the music, they 
had only the words to study. 

Since this study deals with a comparison of specific instructional 
strategies and not learning theory in general, the following references are 
provided for futher reading as an overview of instructional strategies. Gagne 
and Dick (1982) review educational research and its implications for effective 
instruction. Bush (unpublished manuscript) discusses effective instructional 
strategies according to subject matter and student demographics. Weinstein 
and Mayer (1985) describe the application of learning theory in the classroom. 

These citations seem to indicate a need for more empirical data on the 
suggestopedia methodology as well as a need to evaluate the specific 
components of suggestopedia instruction which may enhance learning* 

Description of Instrtxctional^Itethodologies 

^Se treatments are first described and then compared as actually applied 
in the study i Certain modifications were necessitated by the limitations 
imposed by the experimental conditions. Modifications to the treatments 
particularly suggestopedia, were done with the collaboration and approval of 
the contractor for suggestopedia, Lozanov Learning Systems, Inc., and the 
DtlFfcCi 

^ugggs^t43p€dia^^tr uctibnal Methodology. The suggestopedia method is 

designed to approximate many conditions of the early childhood learning 
process. It stresses role-playing, student participant dialogue, and a 
totally positive approach. The instructor (s) encourage interaction of the 
entire class through the use of positive reinforcement, stress relaxation, and 
confidence building techniques demonstrated or modeled by the instructor. The 
classroom environment is also relaxed, comfortable, and non-threatening. 
Subliminal stimuli are also important: the room, furniture, lighting, and 
positioning of class members are carefully selected and arranged to enhance 
student comfort while facilitating student-teacher interaction. 

A phase of directed passivity alternates with an active phase. During the 
passive phase the student listens to material orally presented by the 
instructor with music in the background and without specific directions for 
retaining the materials During the active phase students role play and enact 
situations based on the thematic suggestions from the instructor(s). 
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Suggestbpedia also eaiphaslzes the Importance of the instractor as a figure 
of authority and prestige while providing a relaxed^ comfortable, and 
nonthreatehing classroom environment. 

The soggestopedia method of instruction claims to ensure the acquisition 
of communication skills with emphasis on verbal skills but also including 
reading and writing skills (Sterling^ i984). 

Standard DLIFLC las tructiooal Metiiod^logy^ The methodology described for 
the standard DLIFLC instruction is derived from a Progressive Skill 
Integration (PSI) process which involves the functions^ contents ^ and accuracy 
components of the Interagency Language Roundtable .Skill Level Descriptions 
(see appendiit F for a detailed listing)* (Since a measure of language 
accuracy is uncertain^ especially, at the beginning of the course^ language 
accuracy is always measured in relation to the skill level descriptions and 
not to perfect grammar models tOLIFLC Pamphlet 350^10, 1982]). 

The PSI process uses a functional approach to language instruction that 
stresses the integration of various components of language such as 
prohuhciatloh, grammar, vocabulary, and writing into the co^unication skills 
of proficiency^ The skills of listening^ speaking, reading, and writing are 
?^?^^_^®^®^^P?*^ ^^^P^fh a series of learning activities based upon skill 

acquisition or achievements Exercises such as memorizing dialogues, questions 

and answers, paraphrasing, roie-piaying and translation are 

carried out through a process moving from conceptualization, through 
famiiiartzationj^ and variationi Then a further step to stimulation and 
commanicat ion brings the student to application or proficiency activities* 
Teaching strategies such as audio-lingual (i« e, , speaking-listening) and 
cognitive-code (1. e» , inductive and deductive) techniques are used to 
acconmiodate varying student learning styles* Material is normally presented 
in authentic contexts at normal conversatipnal speeds. The cultural and 
background Information of the language is integrated into the course by using 
various audio-visual training aids* The curriculum becomes increasingly more 
proficiency-based and less achievetaeht-briented over the length of the course* 

Flexible-Scheduling Instructional Hethodolbgy* The flexible-scheduling 
method was similar to the standard DLIFLC method61bgy« (It is not currently 
used in the format described in this study.) The major difference in design 
is that the former used a pacing of the presentation materials based apon the 
Ihstructoris^ assessment of the grbup^s readiness rather than a fixed schedule 
for the presentatibn of materlalSi The emphasis of t^ instruction was on 
speaking through exercises based upon the PSi approach* The weekly training 
schedule was the device used for implementing and monitoring the pace* For 

^^'^f.^^Y^^PP???^ > _?^^^y^®^ function of each 

lesson component in terms of the group's ability to grasp and utilize the 
material presented^ _^aily discussions between instructors allowed for 
schedule changes to accommodate advancement, review, or remediation. Student 
input was incorporated into the schedule. 

i^ntrasts- B etween Methodologies* The physical environment for the groups 
was different* The suggestopedia group had larger rbbms for its sections* 
Room size was smaller and identical for the DLIFLC sect ibns and was 
approximately one-half the size of the rooms used by suggestopedia sections* 
The suggestopedia group had swivel^ high-back chairs and no desks* The DLIFLC 
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grbiips had standard wooden chairs and worked at tablesi the sa^estopedla 
group ased Baroque and contemporary music. No music was used with the DLIFLC 
groups i 

The suggtatopedia group was hot given homework whereas the other groups 
had specific homework assignments. Weekly graded exercises were used with the 
DtiFLG groups but not for the su^estopedia group. Module textbooks were 
provided to all students during ioprocesslng at DLIFLC, but the suggestopedia 
students were told they did not need them and they were seldom used according 
to self reports. 

The suggestopedia group had doe instnictor per section whereas the 
standard DLIFLC group averaged four instructors per day^ with one teaching 
three hours and considered to be the primary instructor. The flexibly- 
scheduled group had two instructors teaching an equal nuniber of hours fo- the 
same daily six-hour schedule as the other two groups. 

Suggestopedia emphasized oial practice with the omiss:.on of raadihg and 
writing skills, particularity during the first- fiVe-weeks. The standard group 
attempted an instructional balance between the three skills, while the 
flexibly-scheduled group attempted the same balance with slightly Sore 
emphasis on oral practices 

Other distiEctions between the standard DLI instructibS and Suggestopedia 
were not as clear. This may be attributed to inconsistencies within the 
methodologies as applied in the classroom as well as to similarities aoong the 
methodologies. Mlgnauit (1978), in his discussion of Lozanov's methods of 
suggestopedia, comparisd Lozaaov's methods with certain critical aspects of 
language instruction and found that suggestopedia incorporated techniques 
noted elsewhere but which have apparently not been c5hsoiidated or adapted for 
use in a unified program of instruction. Examples of overlap include student 
participant dialogues and a positive approach by the instructor (s) . These 
examples are emphasized by the suggestopedia methodology but are also applied 
in varying degrees by the two other methodologies. 

*^^^°P*^^ses on games and role-playing were more Aaracteristic of the 
suggestopedia instructional process than to the DLIFLC instructional 
methodologies. The suggestopedia instruction wai further distinguished by the 
teaching of patterning of information through iStonation, pitch, rhythm and 
proper breathing. It is wOrth noting that, toward the end of the study* the 
suggestopedia instructors were informed that Lozanov had reduced the emphasis 
on these characteristics. This was not considered by the suggestopedia 
instructors as having an effect oh findings from this study. 

Simtlarit les B&tw e eii 4<ethbdolof^ies. The importance placed by 
suggestopedia on the authority and prestige of the instructor did not seem to 
differ from the two DLIFLC groups. This comparability may be an incidental 
consequence of the student population who, as military personnel, are expected 
to respond to teachers as figures of authority and prestige. 

Ail groups were generally presented Materials in authentic contexts and it 
conversational speed. The target language was used as much as possible 
though more so with the su^estopedia group, particularly during the first two 
or three weeks. 
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All groups were administered and evaluated with the same measures of 
academic performance and student attitudes with the exception of the flexibly- 
scheduled groups This group was not initially identified for the 
administration of attitudinal measures due to a lack of certainty about its 
continuation^ and i^^corporation into the study, the effects of testing and 
evaluation on the suggestopedia process of providing a less stressful, more 
relaxed atmosphere was an initial concern. However, the contractor 
representative for suggestopedia and the Ldzanov Learning Systems^ Inc. (LLSI) 
felt that the process and outcomes would not be adversely affected 
(Schleicher, 1985). Measures that addressed proficiency abilities were more 
acceptable to the contractor than the measures of achievement because of the 
instructional emphasis by suggestopedia on proficiency* 

It is important to note at this point that certain influences on^ or _ 
modifications to, the suggestopedia methodblbgy at DLIFLCi such as the effect 
of testing on student stress cited earlier ^ were hot considered by the LLSI 
contract representative to significantly affect achievement ^ prof iciency, or 
attitudinal outcomes. Such mpdificatibhs of the suggestopedia methodology 
considered acceptable included the classroom en^d.ronment^ course length, the 
selection process of student and instructor personnel, testing prbcedares^ the 
degree of iastructpr familiarity vy/ith the testing ihstrumentSj and concurrent 
military duty requirements of the students (Schleicher, 1985). 

Purpbse 

The purpose of this study was to compare the effectiveness of the 
suggestopedia method of instruction with the standard DLIFLC method currently 
used. Ah additional evalution was conducted of a flexibly-scheduled 
methodology, comparing it to both the suggestopedia and standard DLIFLC 
instruction. 



Results from the comparisons of treatment effectiveness werr used, in 
part^ to determine whether the treatments could, wholly or partially, be 
acceptable and/or adaptable for utilization within Army language training 
programs. 

_°^_®9^^^i^y among groups were conducted on descriptive variables 
considered to have a possible effect on treatment dutcomeisi. No differences 

groups were expected because the sample was drawn from a generally 
hoiDogeneous population. 

Tests of significance were conducted among groups on results from measures 
of academic performance. Further analyses were conducted as necessary fbr 
differences between groups and within group effect. Perfbrmance gains for the 
suggestopedia group after 10 weeks of language training were expected to be 
the same as the performance gains for the flexibly-scheduled and standard 
DLIFte groups efter 14 and 15 weeks respectively. TLis expectation was based 
upon the assertion that suggestopedia could accelerate learning (SSC Cbntract 
No. DABT-6d-84-R-b080, 1984). No differences were expected between the 
flexibly-scheduled and standard DLIFLC groups because of the similarity of 
their methodologies and the short difference in time of one week between dates 
for the completion of the curriculum. 
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The suggestppedia group's highest scores were expected to be oh the oral 
communicative skills measured, while the DLIFLC groups were expected to have 
comparable scores on all measures of academic performance. Thesa- expectations 
were derived from suggestopedia's emphasis on oral skills and the DblftC 
groups' instructional balance on each skill tested. 

Ah^halysis of variance was expected to find significant differences in 
student attitudes among groups. Student attitudes from the suggestopedia 
group were expected to be more positive then those found for the DUFLC groups 
because of the emphasis placed upon strong positive student attitudes by the 
suggestopedia method. No diffarehcee were expected between the DIIFLC groups 
tecause of the similarities between the instructional methodologies, 
specifically the uss of the Progressive Skill iSte^ation (PSI) approach. 
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METHOD 



Subjects 

The study Included forty junior enlisted Army personnel scheduled to begin 
the Russian Basic Course (RBC) randomly selected and sorted into two sections 
each for the suggestopedla (h-20) and st&hdard DLIFLC (h^ZO) gr6upvi« One 
section of ten junior enlisted Army and Navy personnel comprised the flexibly- 
scheduled group* This group wa3 p^^^^iously identified and in place prior tc 
its'' incorporation into this study. 

Design 



The study design used matched as^igasnents to the suggestopedia and 
standard DLIFLC methodologies. Tne following descriptive variables were used 
to match the two groups: military rank, military occupatioual specialty (MOS), 
age, years of military service, Defense Language Aptitude Battery (DLAB) 
scores, educational levels prior language training. General Technical (GT) 
scoreSj and gender* Matching was not possible with the flexibly-scheduled 
group because of its later addition to the study* 

The independent variables were the idstructibnal methodologies* The 
dependent variables were the measures of academic performance and student 
attitudes toward their respective methodologies* 



I ns t rumeu t a 1 1 on 

Effectiveness was Treasured by academic performance and student attitudes 
toward their instructional methdolbgies* 

Three measures of academic perf orifiance. were used* One was a set of 

?^i^^yf?f?^ ^^_?9§?^^_§^^§^* ^^^_^^l^^y^™^?^ tests had two major 
components: written and oraii^ to eMmpie of each is p in Appendixes A 

?_» _ ®P???^y?i7* _ Achievement tests are designed to measure performance on 
materials pi^fBented oyer the course of the study (fifteen: weeks}* Five 
achievement tests were administered with each covering a particular set of 
materials presented over different lengths of time* The sets of materials and 
allotted time for presentation were called modules* The modules varied, for 
exatiq^le Module V had more material and required more time for presentation 
than any of the previous four modules* The groups coioi)leted the five tnodules 
according to the respective schedules (i*e*, the suggestopedia group completed 
the five modules in_ten weeks, the flexibly-scheduled group in fourteen weeks, 
and the standard DLIFLC groiip in fifteen weeks)* 

A second measure of academic performance used was the Proficiency 
Advancement Test (PAT)^ a combined measure of both achievement and 
proficiency* The distinction between achievement and proficiency is that 
achievement measures performance on course materials^ while proficiency 
measures performance with the target language regardless of the course of 
instruction^ The PM had three components: listening, reading, and 
speakingi An example of each is provided in Appendixes C, D^ and £, 
respectively* 
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The third measure of academic performance used was a face-td-face oral 
intervlewi fhis was added to the study to obtain a measure of conversational 
proficiency only, which most closely approximates suggestbpedia's 
instructional emphasis, the method used a single blind technique In that the 
two rating instructors did not know which of the three methodologies the rated 
students experienced. 

The measures of achievement cniyj achieyemeut and proficiency, and 

prof iciency only were used in birder to provide for a more balanced and 
comprehensive evaluation of the three methodologies than would be svailablt by 
looking at only achievement or proficiency. 

Student attitudes toward their methodologies were measured usxng weekly 

attitude survuys, a pre- and post test qu'^stionnaire , and an ead-of-course 
ques t ionnair<^4! . 

The weekly attitude survey used a 24-iteiQ semantic differential scale. It 
measured student attitudes about themselves while in class, their opinidmj 
about the class^ and opihlohs about their instractor(s). A sample of this 
instrument is provided as Appendix F* 

The questionnaire adminlsterad at the beginning and end of the study, the 
pre and posttest^ was a 26~item Tiifcert-type scale derived from the work of 
Gardner (personal cbnmunl cat Ion, November i, 1984). It addressed student 
attitudes nbout the potential advantages of learning RussiMi (Part A) ^ and 
student attitudes about foreign isrgoages in geMral (Part B). This 
instrument is provided as Appendix Gi (Further reading on Gardner's 
attitude/motivational scales as related to language learning may be found in 
Gardner, [1983]; Gardner, Clement, Smythe, & Smvthe, [1979]; Gardner & Lalondo 
[1983]; and Gllksman, Smythe, & Gardner [1982].) 

The end-of-couMe questionnaire J as provided in Appendix H, was a 
developmental ins t7-ument, also based upon the work of Gardner, with 126 items 
measuring 12 attitudinai topics using a varie scali» types (l^e*, Llkert, 

semantic-differential, and Tiiultlple-choice). The scales included measures of 
attitudes toward the learning context, student motivation and anxiety, and 
attitudes designed to reflect some key characteristics of suggestopedia. The 
topics and associated scales are described in Table H-1. 

Procedure 



study of the various instructional methodologies was conducted in a 
resident foreign language training environment at the DLIFLG, Mbnterev 
Calif ornia. 

Prior to the study the students received ah orientation from the Directo 
of the School of Russian, D LI FIX, outlining the study. A copy of the 
orientation is provided as Appendix I. In addition, an informal briefing wa 
provided to the experimental groups, suggestopedia and flexibly-scheduled, 
which stressed that there would be no negative effects as a result of any 
substandard academic performance dur-'.ng the course of the study. 

Administration of the instruments consisted of distributing the attitude 

questionnaires, reading the privacy act statement and the standardized 



instructions V and coiiecting the c-^mtiieted questionsialtes; At the beginnriing 
of the study, studeats were adndLiiistereJ the pretest questionnaire. The 
weekly attituiinai survey was adir^inistered at nhe end of each weeS: during 
language laboratory classes or during bre%ks betvreeh classes. All instrucnoro 
di^layed the utart of a class if a student neaded more tiu^ to complete a 
survey. Measures of acadeinjc perf ormance were p^rt of the_ regular class 
schedule. Results were returned after all groups had completed testing with 
the same instrument. The posttes.: and end-of ^course questionnaires were 
administerad during language laboratories at '4:he cbnclutiioa of thi Gtudy 
according to the completion of the Term I (five modules) curriculu?^ for each 
of the methbdolbgies; the suggestopedia and standard DLIFLC methbdotogies 
projected completion dates were weeks 10 and 15 respectively; the f iexibiy- 
scheduled group found that they finished Term 1 in 14 weeks c 

An additibnal assess^roent cf student attitudes was conducted by the DLIFLC 
Office of Organizational Effectiveness at the conclusion of the study and is 
available as a supplementary report (Edwards, 1985). 



RESULTS 



Descriptive Var la bles 

Tests of equality Idund ho significant differences oh descriptive 
variables considered to have a potential effect on treatment oat comes; The 
variables examined are ar follows: military .r?i?h, mi iit£ ry occupatiDnal 
specialty (MOS) J age^ years ol" military service. Def ense Language Aptitude 
Battery (DLW) c^cores^ educat.ibe^l level, prior iaaguage training. General 
Technical (GT) scores, gender, and a pretest measure of student attitudes 
about foreign lahgviages In general; and the potential advantages of leartiing 
lasslanr (A coKparisc:i between protest and posttei^st results may be tDund in 
the section titled, "Me^isures of Student Attitt^des", ) 

The GT score is considered an approximate measure of both verbal ability 
and arithmetic reasoning. Because of the particularly close relationship 
between verbal ability and language learning, tests of group equivalency wvire 
conducted on both components. An analysis of variance (ANOVA) found no 
significant differences among groups dti either measure, 

The DLAB and GT scores are measures of two criteria used for accepting 

personnel into the residence language training program at the DLIFLCi 
Therefore they ware isolated far cbmpaxilsbh with the 1984 student 
population. Tests of equality found the study sample to be representative of 
the 1984 Russian Basic Course Army enlisted student population on both DLAB 
and GT scores. 

Measures of Academic Performance 

Arch ievetne nt^^ea^iures 

Achievement or module tests comprised one of the three types of measures 
of academic performance, A one-way analysis of variance was conducted on the 
two cbmponehts of \;he achievement tests^ written and oral. An additio-ual 
AKOVA was conducted on the \ifeighted exam average. The weighted exam average 
was the higher value assigned, by the DLiFLC, to the written compbhent as 
compared to the oral component when averaging both scores together. (Since 
the completion of this study, the DLIFLC has been in the process of changing 
this emphasis on the written component toward an equal weighting between 
both*) The suggestopedia group had signif icaritly lower scores than the two 
Dti FLC groups on the written and oral cbmpbnents^ and the exam averages. 
There were no differences between the two DLIFLC groups. 

The number of students available for the adinlnistration of a given 

instrument or component therebf was subject to student availability. Student 
availability was affected by such factors as medical problems and changes in 
military duty assignments. Therefore, the number of cases for a statistical 
test varies. 



Written Scores. One of the two main components of the achievement tests 
was the written component. Scores are presented as percentages with a 
possible range of 0-100. As expected the treatment effect was significantly 

11 

25 



EKLC 



different among groups on the written compcnent of the achievetnent tests, 
^2^46) « 48i2l5 25*^^^*_^?^^%"^"^ ^""^^^^s found that the suggestopedia 
vM. * 44i63; sroap did significantly worse than the standard DLIFLC ( « 81.49) 
groop, ^(38) » -8.65, £<.00l. They also scored slgnificantlylbwer than the 
fiexibiy--schsdi;ied » 8d.£7) gr<mp, 4t(27) 6.79^ ^<.001. There were no_ 
significant differences between the two DLiFLC groups^ t(27) « --0.14^ j^.05. 
Subsequent analyses of variance between sections on the written sc^^res 
indicated no within group effect. Appendix B provides a sample of the written 
component of the achisvemenl: tests. 

Oral Scores. The other main component 6^ the achievement tests was the 
oral score. The ANOVA conducted on the tar ee groups Indicated significant 

difference aiDbng groups, F(2,46> « li*43, jK.OOJ. t-tests indicated 

sign?.f icantly iJwev scores for the suggest cpedifi (M^« 62 •90) group }Aien 
compared to the standard DLIFLC » 78*23) groups t(38) * 3i94, jg<.OOU The 
suggestopeviia grocp also scored significantly lower than the flexibly- 
scheduled ^ M 79.93) group, tJ27) « 3 •90,^^01. No differences were found 
between the DLlFtG groups, standard and f iexibiy-scheduied, xX27) =« 0.40, 
o>.05. The analyses of variances for within group effect resulted in no 
signiflcai:\t dif fer«2hcesi Appendix C provides a saaple of the oral portion of 
the achiev&meat tests. 

Exag z\verage . The treatment effect was significant ataong groups, ^(2,46) 
* 4ii50,j»<i001i_ Subsequent t-tests found that the suj;gestopedia 48.19) 
group scored signficantiy lower than the standard DLIFLC (M « 80.83) group^ 
^(38) » -7i96, jg<ibbl. The suggestopedia group also had significantly lower 
scores than the flexibl y-scheduled Qi » 80.68) group,_t(27) » 6.47^ jg<.001. 
No significant difference was found between the standard DLIFLC and flexibly- 
scheduled groups, ^27) « 0.973^ ^>.05. 

A further analysis of variance was conducted using gartley^'s test to 
determine whether the variability of scoreis between sections may have caused 
group differences. There were no significant differences between the two 
sections of the standard DLIFLC control group, E and F, or between the two 
seictibns of the suggestopedia groups I and 5m 

The results in Table 1 reflect the aignificantiy lower scores for the 
suggestopedia group as compared to the DLIFLC groups^ Sections are listed for 
comparisons of similarity of scores within groups, the table also reflects 
the comparability between the DLIFLC groups (standard and flexibly-scheduled) 
on the weighted exam average. 
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Table 1 



Means and Standard Deviations of Module Exam 
Averages by Section, and by Group 






M^ 

— . 


SD 




84.67 


9.53 




76 . S9 


13.45 


E and F) 


80.83 


12.01 




45.62 


10.22 




50.76 


16.90 


and 


43.19 


13.85 




80.68 


8.55 



Section E (standard DLIFtC) 
Section F (standard DblFtG) 
Standard DLIFLG group (avg, of sect. 

Section 1 ( suggest opedia) 
Section J (suggestopedia) 
Suggest opedia group (avg. of sect. I 



Flexibly-scheduled Group (one sect.) 



Suwmary of^^ ladtnss^ The suggestopedia group scored significantly lower 
then the two DLIFLC groups on the written and oral components of the 
achievement tests. The results in Table 2 indicate that the greatest 
difference between the suggestopedia and DLIFLC groups was on the written 
portion of the achievement tests. This finding was expected, in part, because 
of the emphasis placed by the suggestopedia methodology on the oral component 
of language learning. The comparability of written and oral scores for both 
the standard DLIFLC and flexibly-scheduled groups reflects the intent of both 
methodologies to provide an instructional balance between the written and oral 
language skills. 



Table 2 

Means and Standard Deviati ons of Written 

and Oral Module Results by Section, and by Group 



Written Results ^#aX Results 

H SD SD 



Section E 
Section F 

Standard DLIFLC gp. 

Section I 
Section J 
suggestopedia gp^ 

F le xi bl y- s c he du 1 e d 
group (one section) 



85.30 


10.59 


82 . xZ 


6.47 


77.68 


13.02 


74.34 


15.50 


81.49 


12.19 


78.23 


12.23 


41.92 


10.63 


60.80 


9.72 


47.33 


18.00 


64.99 


14.84 


44i63 


14.66 


62.90 


12i40 


80.87 


9.28 


79.93 


5.82 
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Section F (standard DLiFte) 
Standard DLlFLG grottp (avg. of sect. 


E and F) 


H.V9 
80.83 


13.45 

12i0i 


Section t (saggestopedia) 
Section J (saggestopedia) 
Suggest opedia grbap (avg. of sect. I 


and J) 


45.62 
50.76 
43.19 


I0i22 
13i85 


Flexibly-scheduled Group (one sect.) 




80.68 


8.55 



Suwmary o^Jlndings^ The suggestbpedia group scored significantly lower 

then the two DLIFLC groups on the written and oral components of the 
achievement tests. The results in Table 2 indicate that the greatest 
difference between the suggestopedia and DLIFLC groups was on the written 
portion of the achievement tests. This finding was expected, in part, because 
of the emphasis placed by the suggestopedia methodology on the oral component 
of language learning. The comparability of written and oral scores for both 
the standard DLIFLC and flexibly-scheduled groups reflects the intent of both 
methodologies to provide an instructional balance between the written and oral 
language skills. 

Table 2 

Means and Standard Deviations of Written 

and Oral Module Results by Section, and by Group 





Written Results 


Oral Rei 


suits 




M 


SD 


M 


SD 


Section E 


85.30 


10.59 


82. x2 


6.47 


Section F 


77.68 


13.02 


74.34 


15.50 


Standard DLIFLC gp. 


81.49 


12.19 


78.23 


12.23 


Section I 


41.92 


10.63 


60.80 


9.72 


Section J 


47.33 


18.00 


64.99 


14.84 


suggestbpedia gp^ 


44.63 


14.66 


62.90 


12.40 


F lexi bly-8 cheda led 
group (one section) 


80.87 


9.28 


79.93 


5.82 
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Listening Scores ^ No significant difference was found anphg the 
siiggestopedia (M - 58.46), standard DLIFLC = 66.23)^ and f lexibly-schidu Led 
(M » 63i73) groups i ~ 



Reading Scores. Significant differences were found on the reading scores 
among^the three groups, £(2^43) - 6.16, jg^.Ol. Subsequent analyses indicated 
a significant difference between the suggestopedla (8 » 66*715 and standard 
DLIFte m - 73.47) groups, ^<35) -^2.93, je^<.01. A si^ificant effect was 
also found between the suggestopedla and f lexibly-scheduied (K^- 75.10) 
groups, X< 2 6) -2.71, £<.05. No differences were found between the standard 
DLiFtC and flexibly-scheduled groups, t^(25) » 0.63, £>.05. 

An analysis of variance between the sections of the suggestopedla group 
indicated no significahtdifferercesi There were significant differences 
between section E (M - 77.06) and section F (^=. 69.89) of the standard DLIFLC 
group £(16) - -3.13^ i<.01. However, an analysis of variance for the groups 
within treatments design Indicated that the sections did not significantly 
contribute to the variability between groups. 

Spewing Scores The AN0VA found a significant effect among groups on the 
speaking scores of the PAT I, i<2,43) « 7.63, £<.01. 

T-tests between groups found that the suggestopedla ( M =- .64) group scored 
significantly lower than the standard DLIFLC (M^ » .89) group, t(35) * -3i77, 
£^<.01. Scores between the sections within each group were comparable^ No 
significant differences were found between the su^estdpedia and flexibly- 
scheduled (M^-. 79) groups, £(26) - 1.71, £>.P5. Again there were no 
significant differences between the standard DLIFLC and flexibly-scheduled 
groups, £(25) - 1.53, £>.b5. 

The range of scores for the speaking results (D^l) and the oral interview 

scores (0-1) is provided in Appendix J along with the description for each 
range (level). Also provided in Appendix J is the rating expected for a 
language basic course graduate at the DLlFtC. 

F r o f iciBiicy^ M ea s u r e 

Face-to-Face Oral Interview Scores^ The oral interview was the third 
measure of academic performance, it is considered to be a measure of 
conversational proficiency only. 

The analysis of variance of the oral interview scores indicated hb 
differences among groups. These results indicate that the suggestopedla group 
had the same level of proficiency after week 10 that the DLIFLC groups had 
after 14 and 15 weeks of study. 

Summary of Findings 

As TCasures of proficiency were added to the tests of academic 

performance, differences between the suggestopedla and the DLIFLC groups were 
reduced. This is demonstrated by a comparison of differences between the 
suggestopedla and DLIFLC groups oh the PAT results (listening, reading, 
speaking) and the similarity between groups on the results from the oral 
interviews as shown in Table 4. The reduction in differences between groups 
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as measores of proficiency were Increased Is further evidenced, by a comparison 
of the scores from Table 4 with the achievement scores from Table 2. 
(However, the similarity of oral interview scores among groups is most _ 
probably attributable to the low level of proficiency attained after only ten 
or fifteen weeks of study as well as the earlier emphasis suggestopedia places 
upon prof tcieacy as compared to the more gradual development of proficiency by 
the two DLIFLC methodologies i) fable 4 also reflects higher reading than 
listening scores for ail three groups* 



Table 4 

Means aiid Standard beviations of P^T Scores Listening, 
Reading, and Speaking), and Face-to-Face Oral 
Interview Scores by Group 





Standard 


DLIFLC 


Sugges 


topedla 


FlexibljM 






M 


SD 


M 


SD 


M 




Listening 


66^23 


8.57 


58.46 


11.44 


63.73 


9.82 


Reading 


73.47 


5*99 


66*71 


7*89 


75.10 


7.11 


Speaking 


.894 


.170 


i637 


.239 


.789 


.169 


Oral Interview 


.650 


.165 


.674 


.262 


.644 


.167 



S tude nt Attitudes 



Overall^ the at titudinal instruments failed to discriminate among 
groups. Exceptions are noted in the following sectibxis on the weekly attitude 
survey and end-6f-c6urse questionnaire. Results were generally positive 
across groups on each of the three attitude measurement instruments: weekly 
attitude survey, pre- and posttest questionnaire, and end-of ^course 
questionnai re* 

A Z score variable transformation of individual student scores was used to 
standardize results from the Instruments with different observed scales. This 
procedure involved scales from the weekly attitude survey (weeks 1 thru 10)^ 
the posttest, and the end-of-course questionnaire* Ah analysis of variance 
found no significant differences between the suggestopedia and standard DLIFLC 
groups on a measure combining all attitudinal resuitsi £^^ 

2>m05. An ANOVA which included the at titudinal measures available from the 
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flexibly-scheduled group (I.e., the pbsttist and end-of-course questionnaire) 
also resulted in no significant differences among groups, F(2»46) = 0.64, 
j^>.05; * 

Weekly Attitude Surveys^ A ie^kw of the results frbia the weekly attitudi 
survey indicated no differences between the suggestopedia and standard DtlFLC 
groups for weeks one through ten on measures of student attitudes about 
themselves while in class, and student opinions about the class* there were 
no differences between the groups on student opinions about their 
InStructorCs) for weeks five through ten. However, for weeks one through four 
student opinions from the suggestopedia group about their instructor(s) tended 
to be more positive than those from the standard DLIFLC group* 

Pre- and Posttest Q aestionnalre. Results of t-tests found no significant 
differences between pre and posttest Scores for either the suggestopedia or 
standard Dtin,e group. A comparison of posttest scores between the 
suggestopedia and standard DLIFLC groups also indicated no significant 
differences* 

End-of-Coorse Queg^ lQnjia4re. The three groups were compared on results 

from four scales designed to measure attitudes toward key diaracteristics 
alleged to typify suggestopedia instruction: concentration, student- 
centeredness, method, and effort required. There were no differences among 
groups on the concentration, student-centeredness, and method scales. 
However, there was a difference among groups on the effort scale with the 
suggestopedia group indicating a lower level of effort required in their study 
of the target language than the (self -reported) effort required of the two 
DLIFLC groups. Results were comparable between the DLIFLC groups. 

Tw** scales designed to measure anxiety were also evaluated f or group 
differences. A (Russian) class anxiety scale showed no overall differences 
between groups. Oh a (Russian) language use anxiety scale, the suggestopedia 
group indicated greater confidence in their ability to use the Russian 
language than did the two DLIFLC groups* 

There were no ehd-of-course differences amang groups on scales measuring 

student attitudes toward learning Russian, motivational ihteasity, desire to 

learn Russian, attitudes toward their instructor(s) , and attitudes toward 
their (Russian) course. 

On the results from the scale measuring attitudes toward the course 
materials, the flexibly-scheduled and standard DLIFLC groups were more 
positive toward their course materials than the suggestopedia group. 

Student Comments. The importance of the teacher was clearly demonstrated 
across groups by consistently positive student comments. Comments from week 
to week addressed the "friendly", "comfortable", and "positive" approach of 
the instructor(s). The instructor(8) were often singled out by Same for 
praise^ by the students, ".*.*was especially helpful oh verbs", ".i.iis never 
boring . During one period of instructor absence ^ the students noted that the 
quality of the substitute was much lower and they wished they had their 
instructor back as quickly as possible "so that we could learn n>ore". 



17 

31 



Student commehts about the methbdblbgles were also hotewbrthy. During the 
first four or five weeks student coaunents from the suggest ope dia grbup were 
positive and included remarks such as^ "I like it'% "learning faster and with 
confidence"^ and "activities are creative and enjoyable". After the first 
four or five weeks the positive attitudes of the suggestbpedla group were not 
as consistently high as they had been. Self-esteem began to diminish and 
students began to want more cousework structure. At weeks nine and ten of the 
study, when the suggestopedia group was completing their course of 
instruction, student comments became more concerned with their possible 
shortcomings in language skill areas such as "writing" and "granfflnar"^ 



The standard DtiFte group consents du ^J^f ??Y???^ weeks were 

more oriented toward their initial discomfort with the intensity of the 
language training, they felt "nervous", "uptight", "flustered", and a dislike 
for the use of "too niany instructors"^ As noted in the "Contrasts Between 
Methodologies" section, the suggestopedia group had one ^eacher per section 
whereas the standard DLIFLC sections used an average of four instructors per 
day with the primary instructor teaching three of the six hours. The 
flexibly-scheduled group, not addressed in this part of the report, used only 
two instructors in a team teaching approach. 

During the course of the study the standard DLIFLC group became more 
"comfortable", "relaxed", and had more confidence in their instruction, as 
indicated through the studients^ perception of their ability^ and a more 
positive attitude toward their instructors. After the fourth or fifth week, 
commehts centered around materials considered difficult for the students, such 
as "perfect and imperfect verbs". 



As a target of bppbrtunity nbt prigihally cbhsidered ihthe design of the 
study, the flexibly-scheduled grbup'^s comments were bnly solicited after week 
ten. (It should be noted that their academic performance records were 
available and eventually used for evaluation in the study.) These cbmments 
were consistently positive for the five weeks tUey were measured^ weeks 11 
thru 15, especially toward their methodology and instructors. Representative 
cbmments consistently found among the ten students and across the five weeks 
were as follows: 

"I feel that working in a self-paced group has been 
an immense help to me and that because of the 
latitude given to the teachers in order to adopt 
[adapt] their lessons to us, that we were able to do 
better than we could have otherwise." 

"i'^m very certain the best results in teaching us 
Russian is to keep us as a self -paced group and even 
change the rest of the department to that way of 
teaching. We'^re more comfortable this way and our 
teachers know each of our weak points whereas a 
different teacher every hour won't even know our 
names • " 

"(•••••) is one of the best teachers I have ever 
had, she always tries different methods to help us 
learn, and is never bored with teaching. 
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"(;••••) is very good too and is very knowledgeable 
about the customs aiid country." 

Sumary^^f j^indihgs . The results indicated that there were no attitbdinai 
differences among groups^ except as previously hoted^ during the first four 
weeks of attitudes about instructors on the weekly attitude survey^ and on 
three scales used in the end-of-course questionnaire^ the similarity of 
attitudes across groups may be a result of their military background and 
having the same daily military requirements* Differences among groups may 
also have been attenuated because of the similarities in methodologies among 
groups as discussed in the section titled, "Description of Instructional 
Treatments". The short timef rame of the study may have also precluded more 
differences from developing* 

^fi®_P°?l?iy®_3§^i^o^??_f cross groups may have been a direct and/or latent 

result of the positive attitudes of the instructors. The instructors' 
attitudes may have been positive, in turn, not so much as ah effect of their 
methodology but rather the special attention received from the adtuinistrative 
staff and outside observers. This special effect would be more likely 
sustained over the shorter 10-15 weeks of the study rather than the total 47 
week course of instruction. Also, better instructors, who may naturally 
engender positive student attitudes^ may have been selected to e^ioy the 
various methodologies. The likelihood of positive instructor attitudes 
existing and affecting the students is enhanced by the fact that the 
suggestopedia and flexibly-scheduled instructors were also shareholders in the 
instruction because of their involvement in the creation and i^lementation of 
their respective methodology. 

There were two points of particular interest regarding attitudes of the 
suggestopedia group. One was thr± the gradual shift away from the high 
positive attitudes ehcbuhtered during .he first four or five weeks may have 
been a result of instructor "burnout" with the suggestopedia methodology in an 
intensive learning environment, as well as a result of receiving low scores on 
the first achievement test. Student expectations of a more formal student- 
teacher relationship may havc had an impact on their attitudes as a result of 
the actual more informal^ relaxed, and positive approach of the instructor. 
As previously noted in the '^iethod" chapter, students in the suggestopedia 
group were told that grades acquired while in the experiment would not 
adversely effect promotion or class standing. However j it is uncertain what 
real effect this had on the students. The second point worth nbtihjg is that 
while suggestopedia student attitudes were not as uniformly positive during 
the later weeks of the study thiey did remain positive while receiving 
generally low academic performance scores. The standard DLIFLC and flexibly- 
scheduled groups had the sanre level of positive attitudes but also h^d 
generally higher academic performance scores. 

The differences aihohg groups at the end of the course may indicate that 
suggestdpedia's en^hasis on the oral skills does have a positive effect on 
students' comfort with oral communication while the positive attitudes of the 
DLIFLC groups may be tempered by their concerns with the other language skills 
(i.e.^ listening, reading, and speaking). The students' opinions from the 
suggestopedia group about the lack of effort required is most probably a 
result of no requirement to use textbooks and no homework requirements. This 
"lack of effort" perception needs to remain in context with their overall 
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poorer academic performance results as compared to results from the two other 
groups* 

The more positive attitudes of the f lexibly^scheduied and DtlFLC groups 
toward their course materials may be a result of the DLIFLC groups having 
take*-home materials while the suggestopedia group had none* 

DLIFLC Of f ±ce -of-Qrganiza£ional-Ef fectLveness- XGEX^ Assessment of Student 
Attitudes 



An assessment was conducted on student attitudes and their in^ressioris in 
the use of their respective methodologies* The format involved one or two 
interviewers for groups of students ranging from 5 to 12 students per group* 
The overall findings were as follows: 

Students experienced high energy and confidence at 
the beginning of the course [study] * As the course 
progressed and more material was presented^ the 
confidence level rose to what might be called a 
"maintenance level" and seemed to sustain the 
student the remainder of the course [study]* 



The major learning and pedagogical factors 
indicated that the course content and quality of 
instruction were considered to be excellent* The 
suggestopedia methodology was considered to be a 
useful means of [sia] a person to obtain a 
speaking/]i.8tening competency of a foreign language 
in a short period of time* 

the learning situation, e*g., music, soft chairs, 
and absence of distractions, was not reported by 
the students to have a direct effect on the 
enhaiucement of learning a foreign language* 



These conclusions tend to support the findings stated for the attitudinal 
measurement instruments^ especially those from the weekly attitude surveys. 
Further detail from this assessment may be found in a separate report by 
Edwards (1985)* 
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DISCUSSION 



Based upon results from the measures of academic performance, the 

saggesto->edia methodology did riot accelerate learning when compared to the two 
DtlFte p 'ups. In fact ^ the comparison of gains in language learning among 
groups found that those of the suggestopedia group were signif icaLily smaller 
than those of the two DLIFLC groups. The results indicating a significant 
difference were from both the written and oral measures of the achievement 
tests, and on two of the three measures (reading and speaking) from the 
Proficiency Advancement Test (PAT)* 

Though the suggestopedia group had ginerally lower scores than the DLlFbC 
groups on the measures of academic performance, test results indicated that 
there were smaller differences between the suggestopedia and DUfLC groups as 
measures of proficiency were added to the evaluation of student peiformaSce. 
This trend is noted by a review of thg results from the PAT indicating less 
differences between the suggestopedia and DLIFLC groups than were found on 
results from the achievement tests i Additiorially, the face-to-face oral 
interview findings reflecting measures of proficiency baly, showed no 
differences among groupsi Therefore, the comparatively lower scores of 
suggestopedia found on achievement measures during the early stages of 
language learning when suggestopedia is emphasizing proficiency and the DLIFLC 
groups are emphasizing achievement is not surprising. However, It is 
important to note that evaluations of proficiency during the early stages of 
language learning may not be able to provide clear discriminations between 
methods of instruction because of the low level of proficiency available to 
the students. 

The combination of findings from this study^ (i.e., to include the 
positive student attitudes within the suggestopedia group despite low academic 
performance, and their confidence with Using the target iaSguage), along with 
the research In related suggestopedia areas ^ Indicate a potential use for at 
least some of the components found in the suggestopedia method. Examples or 
potentially useful components of euggestopedia include the following: the use 
of incidental learning as a teaching technique; the eiphasis on relaxation and 
a p^itive attitude toward the target language; and the attempt to immerse the 
studen^. into the target language. 



^f^°if^^*=°Ped±a were to be used as some form of enrichment adjunct to the 
established DLIFLC or other military language training programs, it should 
probably be used from one to five weeks. This may be the best time interval 
since positive student attitudes and the instructor energy required to 
li^lement a suggestopedia progratn seemed to peak and then diminish after 
approxlmateiy five weeks of intensive application. The effect of inscructor 
energy was reported by the suggestopedia instructors and noted in the 
Schleicher report previously cited. Those components listed in the previous 
paragraph that are found useful could be incorporated throughout a course of 
instructions 



The evaluation of the flexible-scheduling methodology indicates that the. 
can be a time savings over a course of instruction by the instructor(s)'close 
monitoring and supervision of student progression through a curriculum. The 
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time saved may be used for review, for the presentation of new material^ or 
for early advancement to additional training or a field assignment. Both the 
resales from measures of academic performance and attitudes indicated that the 
PP^i^*^® f^^"^^?8s from both measures were sustained over the course of the 
study. In fact, an informal follow-up evaluation of the flexibly-scheduled 
group, conducted approximately one month later (to determine whether the 
i^ethodology should be continued throughout the 47-week course)^ Indicated that 
they had maintained their lUgh levels of performance and positive attitudes. 

This report is based upon findings which are ^ iii part ^ a result of a 
specific experimental design characterized by such elements as the time 
covered by the study, the population sampled, the sample size, differences in 
irstructorsj and the lost nixnents employed. Conclusions drawn from this report 
should not be generalized outside the context of this study. 
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APPENDIX A 
WRITTEN COMPONENT (ACHIEVEMENT TESTS) 
MODULE 1 



PART 1 LISTENING COMPREHENSION 

Stimulus Twenty Russian sentences spoken on tape; 

Task Select correct translation of word or phrases in each sentence. 

Options Four English options. 

Concent Vocabulary and grammar from Module 1. 

No« of Items 20 

PART II DiCTAfiON 



Stimulus Russian sentences spoken twice on tape. 

Task Write. the words that have been omitted on student^s sheet. 

Content Vocabulary and gramnar from Module 1. 

No. of Items il 

PART III TRANSLATION 



Stimu lus Written English sentences. 

Twsk^ Translate in writing each sentence into Russian. 

Use of lexical aids is not permitted. 
Content Vocabulary and grammar from Module 1. 

No. of Items 10 
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MODULE II 



PART I 

gtl jmiliis 

Task 

Content 

Nb« of Items 

Stlmaius 
Task 

Content 

No. of Items 

PART III 

Task 

Content, 
No* of Items 



DICTATION 

Russian sentences spoken twice on tape. 
Write down the sentences in Russian* 
Vocabulary and grammar from Modules i and li 
(emphasis on Mod* II)* 
7 



LISTENING Ce>ffREHENSION 

Two short Russian passages spoken on tape once. 

Answer in English in writing questions which are written in 

studcnt'^s copy of the exam. 

Simpie passages based on the vocabulary and grammar of Module 

and II (emphasis on Mod. II). 

8 



TRANSLATION 

English sentences written oh student 's . sheet. 
Translate in writing each sentence into Russian. 
Use of lexical aids is not permitted. 
Vocabulary and grammar from Modules I and li. 
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MODULE III 



PART I DICTATION 

Stimulus Russian sentences spoken twice on tape. 

Task Write down the sentences in Russian. 

Content Vocabulary and graznmar from Modules I-lII 

(emphasis on Mod. III). 
No. of Items 10 



PART II LISTENING COMPREHENSION 



Stimulus Two short Russian passages spoken on tape once^ 

Task ^^swer in English in writing questions which are written 

in student's copy of the exais. 
Content Simple passages based on the vocabulary and grammar of 

Modules I-III (emphasis on Mod. III). 
No. of Items 9 

PART III TRANSLATION 

Stimuli English sentences written on student '^s sheet. 

Task Translate in writing each sentence into Russian. 

Use of lexical aids is not permitted. 
Content Vocabulary and grammar from Modules I-III 

(emphasis on Mod. III). 
No. of Items 9 
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MODULE IV 



PART 1 DICTATION 



Russian sentences spoken twice on tape« 
Write down the sentences in Russian. 
Vocabulary and grammar from Modules I-IV 
(emphasis on Mod, IV). 
10 



Stimuius- 

Task 

Content 

No. of Items 



PART II 



LISTENING COMPREHENSION 



Stimulus 

Task. 



Content. 

No. of Items 



Two short Russian passages spoken on tape once. 
Answer in English in writing questions which are 
written in student's copy of the exam. 
Simple passages based on the vocabulary and grannar of 
Modules I-IV (emphasis on Mod. IV). 



PART III 

S^TOlluS 

Task 

intent 

No. of Items 



FILL-INS 

English words in Russian sentences. 
Translate in writing each word into Russian. 
Use of lexical aids is not permitted. 
Vocabulary and grammar from Module IV. 
i2 



PART IV 

Stimulus 
Task 



Content 

No. of Items 



TRANSLATION 

Written English sentences i 

Translate in writing each sentence into Russian. 
Use of lexical aids is not permitted. 
Vocabulary and grammar from Modules I-IV 
(emphasis on Mod. IV). 
10 
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MODULE V 



PART I 



DICTATION 



Task 
Coot eat 



No* of Items 



Russian sentences spoken twice on tape* 
Write down the sentences in Russian. 
Vocabulary and granmar from Modules I-V 
(emphasis on Mod V). 
6 



PART II 



LISTENING COMPREHENSION 



Stimulus 
Task 
Options 
Content 



Short statements in Russian spoken on tape. 
Select appropriate response to fit the situation. 
Three English options*. 

Simple situations involving statements based on 
Modules I-V (emphasis on Mod. V). 



No. of Items 5 



PART 111 



Frt,L-INS 



Stinialus 
Task 



Content 

No. of Items 



English words in Russian Sientences. 
Translate in writing each word into Russian. 
Use of lexical aids is not permitted. 
Vocabulary and grammar from Module V. 
8 



PART iV 



TRANSLATION 



Sttmulos 
Task 



Content 



No. of Items 



Written English sentences i 

translate in writing each sentence into Russian. 
Use of iexicai aids is not permitted^ 
Vocabulary and grammar from Modules 1--V 
(emphasis on Mod. V). 
9 
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APPENDIX B 
ORAL G0MPGNEKT (AefllEVEMENT TESTS) 
MODULE 1 



PART I 



READING 



St imulu s 

Task 

Coatant 



No* of Items 



Short passage in Russian (33 words). 
Read the text aloud* 

Simple Russian text based on the familiar and unfamiliar 

Vocabulary* 

1 



PART II SPEAKING 

Teh information questions asked in Russian by the instructor* 
The questions are based on the material from Module 1* 



B-i 45 



MODULE II 



PART I READiNS 

Short passage in Russian (about 40 words). 
Read the text aloud, 

Simpie Russian text based on familiar and unfamiliar 
vocabulary. 
1 



Stimulus 
Task 
Content 

No« of items 



FiM 11 QUESTIONS 

Ten information questions asked in Russian by the instructor. 
The questions are based on the material from Module II. 

PART III TASK 

One^task written in English to elicit an oral response based on the material 
of Module II. 
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MODULE III 



PART I QUESTIONS 

Ten information questions asked in Russian^b^ ins tractor, 
the questions are based on the material from Module III* 



PART II TASKS 



Two role-playing situations written in English to elicit an oral response 
based on the material of Module III. 
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MODULE IV 



PAST I QtJESTiONS 

Teh information questions asked In Russian by the instructor. 
The questions are based on the material from Module IV. 



PART ii TASK 

One task written la English to elicit 
of Module IV. 



an oral response based on the material 



ERIC 
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MODULE V 



PART i (^STIONS 



fen information questions asked in Russian by the instructor, 
the questicns are based on the material from Module V. 

PART II TASK 

One role-playing situation written in English to elicit an oral response based 
on the material of Module V* 
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APPENDIX C 

LISTENING COM? EtEHEN SIGN, PROFiCIENCY ADVANCEMENT TEST (I) 



1. PART ONE (WORDS AND PHRASES) 

Stimulus - Sentences/phrases spoken on tape^^ 

Task - Select correct tratxsiatioD of word or phrases In each level. 
Options - Four English options. 

Content- Familiar high frequency expressions, frozen phrases. 
No. of Items - 25 
Level - 0+;i 

2. PART TWO (SITUATIONAL RESPONSE) 

Stimulus - Sentences spoken on tape. 

Task ^ Select appropriate response to fit situation. 

Options Four target language options. 

Gontent - Simple situations involving very simple statements and 

responses. 

Noi of items - 15 

Level - 0+/i 

3. PART THREE (GIST I N§/ SHORT PASSAGE) 

Stimulus - Sentences or short paragraphs spoken on tape twice. 
Task - Select correct summary of passage. 
Options - Four English options^ 

Content - Simple passages reduced tc tevei i^l^ wordxngi 

Words in multiple choice options will help furnish context; glosses in 

English may ba used if absolutely necessary. 

No. of Items • 15 

Level - 1+/2 

4. PART FOUR (FACTUAL QUESTIONS /LONGER PASSAGES) 

Stimulus - Four or five passages (50-7C words) spoken on tape twicer 

Task - Answer six or seven factural multiple-choice items on each passage. 
Options - Four English bptibxis. 

Content - Simple paissages reduced to LeVel 2 wording with very minor 
editing; words in multiple choice options will help furnish context; 
glosses may be used if absolutely necessary. 
No. of items - 23-30 
Level - 2 



total No. of Items - 85 
Range - 0+/2 
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APPENDIX b 

READING COKPREKENSION, PROFICIENCY ADVANCEMENT TEST (I) 



!• PART ONE - (SIGNS AND EXPRESSIONS) 

Stlinulus Printed signs ^ frozen/memorized phrases. 
Options - Tour English options. 
Number of Items - 10 
Level - 0+ 

2. PART TWO (VOeABUtARY IN CONTEXT) 

Stimius - Short printed sentences with underlined word, 
task - Choose translation of underlined words. 
Options - Four English options. 

Context - High frequency/familiar vocabulary in familiar context. 
No. of Items - 15 
Level - b+/l 

3. THREE (eONTE3tTOAL CObffREBENSlON) 

Stimulus - f^^r short sentences or paragraphs with blanks, 
task - Choose correct form to fill blank. 
Options - Four target language optioTOi 

Content - High frequency/familiar granmar patterns and forms. 
No. of Items - 15 
Level - 0+/1 

A. PAilT FOUR (QUESTIONS ON PASSASE/GISTING) 

Stimulus - Two/three short passages (45-70 words)* 

Task - Answer multiple-c5ioice questions (factual summary). 
Options - Four English options. 

Content - Authentic* passages reduced to Level i+/2 wording. 

Multiple-choice options will help furnish context: glosses in English may 
be usedif absolutely necessary to furnish context. 
No. of Items - 12 
Level - 1/1+ 



*( In these documents the word "authentic" refers to published written material 
in the target language outside the course of instruction.) 
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PART Fj-VE (cloze PASSAGE) 



Stimulus - Short passage (80-126 words) with paraphrase. 
Task - Select options to restore passage. 

Qptidns - 3-4 symBols, 12-16 deletions, 10-15 options under each iyibSl. 
Content - Simple authentic passage with very little editing; paraphrase 
will be used to furnish context. 

glosses in English may be used if absolutely necessary to furnish context. 
No. of Items - 16 
Level - l*/2 

PART SIX (INFORMATION IDENTIFICAHON) 

Stimulus - Short passage (125-150 words) with underlined words and 
phrases. 

Task - Selec^ iuitlple-choice option that translates uSderiined portion. 
Options - 4 multiple-choice. 

Content - Simple authentic passage wich very little editing; glosses in 
English may be used if absolutely necessary to furnish context. 
Noi of Items - 17 
Level - 1+/2 



total Noi of Items - 85 
Level Ran.ge - 0+/2 
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APPENDIX E 

SPEAKING COMPONENT, PROFICIENCY ADVANCEMENT TEST (i) 



1. PARI ONE 

Three Information questions designed to elicit speech at levels 0+ 
through 1. 

2. PART TWO 

Two role-playing situations designed to elicit speech at levels 0+ 
through 1. 
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WEEKLY ATTITUDE SURVEY INSTRUMENT name 

p^jyacj^ - Act - St at emeat 

SSAN 

You are being aske^ to complete this questionnaire as part of a research 
effort which seeks to find ways of improving both teaching and learning at 
DIiIFLC. Disclosure of requested information is solicited under the authority 
of Title 10, United States Code 3012, Executive Order 9397, and Army Regulation 
6OO-2, and is voluntary. Failure to provide name or SSAN may result in 
misidetltification arid thUs jeopardize the firidirigs arid conclusions of this 
study, with possible negative impact on the Defense Foreign Language program 
as a whole. 



Your ariswers to all questions will be held iri strict confiderice^ arid your 
riame and SSAN are requested briJ^sp that yoUr ariswers can be associated with 
other informatibri that is esseritial to this research effort and is coritained in 
ybur official recbrds arid trairiirig files. Neither ybur iristructbrs nbr yoUr 
sUperibr officers ripr anyone else other than the researchers will be able to 
associate ybUr identity with yoUr respbhaes bri this questioririaire, and everi 
the researchers will use assigned numbers to refer to ybur questionnaire data 
arid to all bther iriformatiori that is collected as part bf this research. 
Student responses will be reported prily as aggregates (e.g., "20?t of the students 
thought that....") or as anonymous individuals (e.g.^ "one student commented 
that ....). 

Iri order for the results to """e meaningful ^ it is important that ybur 
ariswers be accurate and frank as possible. Also, the usefulness bf your 
quest ibnriaire will be lessened to the extent that you do not answer each item. 
Therefbre^ you are urged to answer all items unless it is very important to 
you personally to omit certain ones, in which case you may omit them and go on 
to the others. If you have questions about any items, please raise your hand 
for assistance. 
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ouiici. cuuiiaenciaxicy wij-j. De matntatnedi 

_ ^ Fd^ each pair of words plici in "x" in the space that best reflects what y5u 
feel and how strongly yoa feel about the Idea conveyed by each word-pair. 

ni^^^^ll^i.^^- w°*^-Pfp8 below, describe your actual feelings whi le in t his 
class (that is. describe yourself while in class) during the palT week: 



comfortable 

attentive 

uncertain 

positive 

relaxed 

trusting 

bored 

insignificant 

happy 

calm 

smart 

depressed 



uncomfortable 

restless 

confident 

negative 

uptight 

guarded 

stimulated 

significant 

unhappy 

flustered 

stupid 

excited 
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th« ifoliovibf ett of word-pairs to d«icrib« jour opiolont about thm 
i duriag ihm piit wick* 



llk«d 

tatcreited 

easy 

dreaded 

tedloua 

pbiitive 



diaiikad 

bored 

hard 

enjoyed 

f aiciuatltig 

negative 



Uie the following yord-palra to describe your oplolona about your 
instructbr(i) during the past week. 



coiipetent 
caring 

unaure 

d^ii 

un coaf or table 
creative 



Any other coamehts you would like to sake? 



:ent 
inaenaitive 
confident 
exciting 
relaxed 
uncrcative 
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PRETEST AND POSTTEST QUESTIONNAIRE name 

Privacy Aoi Statement 

SSAN 

You are being asked to compiete this questionnaire as part of a research 
effort which seeks to find ways of improving both teaching and learning at 
DtlFte. Disclosure of requested information is solicited under the authority 
of Title 10 i United States Code 3012, Executive Order 9397* and Ar^ Regulation 
600-2, and is voluntary. Failure to provide heune or SSAN may result in 
mis identification and thus jeopardize the findings and conclusions of this 
study i with possible negative impact on the Defense Foreign Language Program as 
a whole. 

Your answers to ail questions will be held in strict confidence ^ and your 
name and SSAN are requested only so that your answers can be associated with 
other information that is essential to this research effort and is contained in 
your official records and traihihg files ^ Neither your instructors nor your 
superior officers nor anyone else other than the researchers will be able to 
associate your identity with your responses on this questionnaire, and even the 
researchers will use assigned numbers to refer to your questionnaire data and 
to ail other information that is collected as part of this research. Student 
responses will be reported only as aggregates (eigi , "2056 of the students 
thought that...i") or as anonymous individuals (e.g., "one student commented 
thatiii.")i 

in order for the results to §e meaningful^ it is important t^at your 

answers be accurate and frank as possibiei Also; the usefulness of your 
questionnaire will be lessened to the extent that you do not answer each itemi 
Therefore, you are urged to answer all items unless it is vary important to you 
personally to omit certain ones, in which case you may omit them and go on to 
the othersi If you have questions about any items, please raise your hand for 
assistance. 
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PART A 

The following stateients are about potential advantages of learning Russian, 

However^. you may not agree that a given statement has any relevance to 
personally . For example: ^ — 

"I like having the opportunity to learn Russian because I will be 
able to read Russian literature in the original." 

^If reading Russian literature is totally irrelevant to you, and you can't 
imagine why anyone would want to, you would write 1 for strongly disagree. On 
the^ other hand if reading Russian literature in the original is one of your 
most important reasons for learning Russian, you would write 6 for strongly 
agree.^ Of course, your response to this statement may lie somewhere between 
those two extremes; in that case, yoU vould write 2, 3, U, or 5. 



Agree Agree Disagree Disagree Disagree 
Strongly Moderatel3^ ^Lightly Slightly Moderately Strongly 
6 5 5 3 i — i 

I like having the opportunity to learii Russian because it will..., 

1* ^^Ip S^t a job after I have completed my military service. 

2. increase my ability to influence others. 

3. enable me to better understand Russian cxilture. 

. ^» make me a better educated person. 

— 5. give me an edge in competing with others. 

6. advance nsy military career. 

?• enable me to get to know Russians. 

_ 8. help me understand what Russians really want. 

9» make me appear more cultured. 

enable me to meet and converse with a greater variety of people. 

11 • Iielp me earn a college degree. 

12. enable me to interact socially with Russians. 

_ 13. help me get the kind of job I want in the militai^. 

Ik. help me protect my interests in dealing with Russians. 

15. help me qualify for additional training in the iilitary. 

■"— 16 • make me more attractive to future employers. 
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PART B 

This part of the ciuestlonnaire asks you to indicate how much you agree or 
disagree with a series of statements about foreign languages In general. Your 
opinions or feelings may lead ydU to agree with some statements and disagree 
with others. There are no right or wrong answers--Just your point of view. 



REMEMBER: 1. It Is ybur honest opinion that Is being requested, and 

your responses will be treated with strict confidentiality. 
2. Mark each statement according to your first Impression; 
It Is riot necessary to take a lot of time for any one 
(luestlon. 



Please read each statement carefully and write In the number that best describes 
your response. 

Agree Agree AS^ee Disagree Disagree Disagree 

Strongly Moderately Slightly Slightly Moderately Strongly 
t 5^321 

1. I Would really like to learn many languages. 



2. When I see a foreign film, I would rather hear the sound track 
Iri English than to hear the original language and see English 
subtitles. 

3. Foreign languages sound like gibberish to me. 

km Knowing -foreign languages can help one convey many feelings and 
Ideas that are not easily expressed In English. 

5. I wish I could speak several languages fluently i 



6. If I planned to live in another country, and I thought i could 
get along in English, I would not make such effort to learn the 
language. 



7. I really have little interest in foreign languages. 

8. I often wish I could read newspapers and magazines in many 
languages. 



9. I enjoy meeting and listening to people who speak other languages. 
10. Studying a foreign language is not a pleasant experience. 
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NAME 



DATE 

APPENDIX H 
END-OF-COURSE QUESTIONNAIRE 

Now that the methods comparison experiment in which you have been 
participating has come to a close, we would like to ask you to help us evaluate 
it. 

You are being asked to complete this questionnaire as part of a research 
ejTfort which seeks to find ways to improving both teaching and learning here at 
DLI. Your answers to all questions will be treated in strict confidence. We 
aok for your name only to associate your i*esponses on this questionnaire with 
other information of importance to our r'^search. We will immediately assign 
nijmbers to your questionnaire and all other infqnnation we collect and only the 
numbers will be Tised in our analysis. Neither your instructors nor anyone 
else at DLI or in the military will have access to any of these questionnaire 
responses except as aggregated in research findings (e^gi ,"20?& of the students 
felt that....). The researchers will be the only persons who will see the 
questionnaire with your name on it. 

For the results to be meaningful, it is i^ortaht that you be as accurate 
and as frank as possible in your answers. If you do not want to answer any 
particular item, you do not have to. However, you should realize that the 
usefulness of your questionnaire will be lessened to the extent that you do not 
answer each item. We, therefore, urge you to answer all itfems unless it is 
important to you personally to omit certain ones. Ir you have difficnltieg or 
questions about any one of the items, please raise your hand and someone will 
come to your assistance. 

Thank you very much for your cooperation. 
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Section I 

For each of the foiioving statements, indicate the number from the foiiowihg 
scale which best represents your reaction to the statement. 

Disagree Disagree Disagree Agree Agree Agree 

Strongly Slightl y Slightly Btrongii 

3 k 3 — — 

i» I enjoyed participating in class, 

2. I found that the course required extraordinary effort on my part. 

3. i^y mind often wandered wheti I was in class. 

^» I^* I had it to do over again, 1 would avoid studying Russian. 

5» i did not get anxious when I had to respond in class. 

6* I would feel calm and sure of myself if I had to order a meal in 

Russian. 

7. I would have preferred another method of teaching this course. 

8» I often felt uncomfortable in class i 

9» if ever 1^ should run into a group of people speaking Russian, 

i would feel relaxed in Joining them. 

10. I found the instructors responsive to my particular learning 

heeds. 

ii» Most things we learn in Russian are interesting. 

■ 12. I really couldn't understand people who got uptight about usi.ig 

Russian in class. 

i3. It was easy to remain attentive in this class. 

So far, I have found this program easier than I expected. 

3:5 • I really enjoy learning Russian. 

16» I would feel uncomfortable speaking Russian in any real-world 

situation. 

^ 17. Is^y instructor(s) seemed genuinely interested in my progress in 

this course. 

18 • 1 never felt quite sure of myself when I was speaking in class. 

19 • I learn more when other teaching methods are used. 

20. In all honesty, I -^ould rather do almost anything that study 

Russian. 

21. This was one of the most demanding courses I have ever taken. 

22. i felt confident when active participation took place in class. 

23. I would feel comfortable speaking Russian in an informal 

gathering where both English and Russian speaking persons were 
present. 

2U. I plan to continue my study of Russian after I complete this 

course. 
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Disagree Disagree Disagree Agree Agree Agree 

Sto ^ongly -- Slightly Slightly Strongly 

^1 2 3 5 5 6 

25. VJhen in class, I was completely absorbed in what was going on 
in the classroom. 

26. To be honesty I hate Russian. 

27. Ity instructors were adequately sensitive to the problems of a 
beginning student. 

28. I often dreaded going to class. 

29. I am sure I would get nervous if t had to speak Russian to a 
sales clerk. 

30^ LeaiTiihg Russian takes so long, the attempt does not seem 
worthwhile. 

31. It embarrassed me to volunteer answers in class. 

32. I usually had plenty of time to complete all class assignments. 

33. I would feel confident and relaxed if i had to ask street 
directions in Russian. 

3k. i am glad to have the bpportxxhity to learn Russian. 

35. instructorCs) seemed determined to cover specified material 
regardless of student readiness i 

36. i often had difficulty keeping my attention focused on cl8.ss- 
room activities. 

37. I think learning Russian is boring. 

38. it bothered me that the other students spoke Russian in class 
better than 1 did. 

39. I would get flustered if it were necessary to speak Russian 
when making a telephone call. 

UO. I would recoEttmend that this course always be taught using this 
method • 

kl. Making a hotel reservation in Russian would bother mei 

k2. The satisfaction of learning Russian makes the effort worthwhile. 

U3. I was generally relaxed in class. 

kk. This course seemed "better geared to the way I learn than other 
courses I have taken. 
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Section II 

oo^oifif^^^J^^?" °f the following items by circling the iitt« of the 

completion which appears to be most applicable to you. Please be ai f^ank and 
accurate as possible, the proper evaluation Of this experiment depindi upon it. 

45. During off-duty hours, I thought about what I had learned in my Russian 
C • • • • 

a • very frequently 
b» seldom or never 
c. once in a while 

46. W^eh I had a problem understanding something we were learning in Russian 
class, I would usually. i.. ^ 

a. ask the instructor for help 

b. seek help only just before the exam 

c. just forget about it 

47. When working oh assignments, 1 usually,..^ 



a 



49. 



put some_ effort into them^ but not as much as 1 cocld 

b. worked very carefully, making sure I understand everything 

c. did as little as possible 

d. [Not applicable — we had no assignments.] 

48. Considering how i studied Russian, I can honestly say that I.... 

a. did as much work as the next person 

b. will pass on the basis of sheer luck or intelligence, because 1 did 
very little work 

c. worked ve^ hard to learn Russian 

d. [Not applicable~we were hot required to study outside of class.] 

If my instructor wanteu someone to do an extra Russian ass^'gnment, I 
woula .... 

a. definitely not volunteer 

b. definitely volunteer 

c. do it only if the instructor asked me directly 

d. [Not applicable — ^we had rib assignments.] 

50. After I got my Russian assignments back, I usually.... 

a. reviewed them, correcting my mis-cakes 

b. just put them aside and went on 

q. looked them over, but didn't bother correcting mistakes 
d. [Not appiicabie~we had no assignments.] 

51. When I was in Russian c:lass, 

a. volunteered answers as much as possible 

b. answered only the easier questions 

c. hardly ever said anything 



52. When there was an event involving Russian language or culture, I usual! 
went 



a. even if i had to pay 

b. only if it was free or subsidized 

c. orly if required to do so 
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53. I made a point of talking to thy inistructors outside of class.ii. 

a; only if l was having difficulty understanding something 
b; to learn as much as I could 
c; not at all 

54. i used the Russian materials in the Learning Resources Center (films, 
tapes ^ etc. ) . . . . 

a. ?S9?la^ly 

b. seldom or never 

c. once in a while 

55. If I thought by staying in the military I had a good chance to take 
intermediate and advanced Russian.... 

a. 1 would definitely go for it 

b. it would make no difference in my plans 
ci I would get out as soon as possible 

56. I want to learn enough RuisJsian. . . . 

ai to get through the course 

bi to be really fluent 

Ci to do my job satisfactorily 

57. The amount of Russian I have learned so far.... 

a. ir enough to turn me off completely 
bi is satisfactory; i can get by with it 
Ci has just whetted my appetite for more 

58. The further along I go in my Russicui studies, the more I.... 

a. wish i had started long ago 
hi want to keep on going with it 
Ci wish it were all over 

59. During Russian class/ I would likei..i 

a. to have a combination of Russian and En^li/^h spoken 
hi to have as vmch English as possible spoken 

c. to have only Russian spoken 

60. If I had the opportunity to speak Russian outside- : class, I would.... 

a. never speak it _ _ 

b. speak Russian most of the time, using English ! i.t really nc-rsssary 

c. speak ±t occasionally 

61 « If it were strictly up to me whether or not to learn Rix^^f-lan^ I.... 

a. would definitely tcike it 

b. would hot take it 

Ct don't know whether I v/ould take it or not 
62. I find Studying Russian. ... 

a. not interesting at all 

b. as interesting as most other siibjects I've scudied 

c. especially interesting 
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63. If I had the opportunitj I would watch Russian TV and listen to Russian 
radio i • ^ i 

a. sometimes 

b; as often as possible 
c • never 

6h. If they were available, I would read Russian iagazines and newspapers..., 
a; as often as I could 

b. seldom or never 

c. not veiy often 

Section III 

The scales that rollow attii^t to capture your overall impressions of the 
instructors, course materials, and course in general, to whic!^ you have Just 
been exposed. You will be asked to rate each of these three things on a serie 
of scales, each of which consists of a pair of words expressing opposites with 
seven spaces in between. For example, assume you are rating the course on thi" 
scale : 

1 2 3 U 5 6 7 
^^^"^ : : : : ; ; easy 

If you place an "x" in the "1" position, it means that you found this course 
to be extremely hard; an "x" in the "7" position would mean you found it ex- 
tremely easy. Position? 2-3 and 5-6 are used to indicate gradations of these 
opinions, and position k indicates that you found the course neither hard nor 
easy, but somewhere in the "average" ranges 

in answering thi« part of the questionnaire, please work quickly and do 
not stop to think about each scale. It is your immediate impressions in which 
we are interested. 

A. iNSfRUeTOR(s) 

V J?^^^''^ Pfi^ of words c:i this scale, place an "x" in the position which 
best fits j'our Impression of your ]pT±^ry_ inr tructortsj^ . 

i 2 3 ii r 6 7 

efficient : 

insensitive : ... 

cheerful ■ . . . . 

• *• • • : 

competent : . . , 

insincere 

unapproachable : . 

pleasant • 

trusting : . ^ 

incapable ; ^ - . . , 

tedious - . . , 



inefficient 
sensitive 
U«erless 
iiicompetent 
sinjeie 
approachable 
unplc5.j?aht 
suspici-^us 
capable 
fascinating 
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1 2 



3 



6 



7 



f rjiendly 

exciting 

organized 

unreliable 

unimaginative 

impatient 

polite 

colorful 

unintelligent 

good 

industrious 

boTring 

de;-;endable 

disinterested 

inconsiderate 



unfriendly 
dull 

disorganized 

reliable 

i maginative 

patient 

impolite 

colorless 

intelligent 

bad 

unindustrious 

interesting 

undependable 

interested 

considerate 



B. COURSE MATERIfttS 



For each pair of words on this scale, place an "x" in 'die position which 
best fits your impression of the course materiala you usedi 



important 

meaningless 

dull 

interesting 

organized 

valuable 

confusing 

hard 

natural 

irrelevant 

clear 

useful 



1 



7 



unimportant 

meaningful 

stimr^lating 

bbringf 

disorganized 

worthless 

coherent 

easy 

artificial 

relevant 

uhi rite Hi gib le 

useless 
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- Fpr each pair of words on this scale, place an "x" in the position which 

best fits your impression of the course as a whole as you experienced it during 
the weeks in which the experiment was in progress. 



1 



meaningful 

enjoyable 

monotonous 

effortless 

awful 

interesting 

good 

simple 

disagreeable 

fascinating 

worthless 

necessary 

appealing 

useless 

elementary 

pleasurable 

educational 

unrevrarding 

difficult 

satisfying 

unimportant 

pleasant 

exciting 

clear 

colorful 



meaningies s 

unerijoyable 

absorbing 

hard 

nice 

boring 

bad 

complicated 

agreeable 

tedious 

valuable 

unnecessary 

unappealing . 

useful 

complex 

painful 

noneducational 

rewarding 

easy 

unsatisfying 
important 
unpleasant 
dull 

confusing 
colorless 



THANK YOU FOR ALL YOUR COOPERATION I 
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APPENDIX I 

SCRIPT FOR STUDE^^T ORIENTATION ON EXPERIMENT 



Some of yoa may wonder how a particular method of teaching or training is 
approved and authorized. Since DLi is responsible for conducting language 
training, developing course materials for use here and in the fields 
developing tests for use nere and throughout ^0© ^prof iciency tests and 
aptitude tests — DLPT/DLAB)^ for language training standards and f research 
in Foreign Language (FL) training, we constantly need to evaluate these 
efforts and attempt to improve them. 

Newly developed materials and tests, for example, are validated by 
actually using them with student groups, then by analyzing and studying the 
results. Sutsequently , any revisions which may be needed are made in an 
effort to continue improving our courses and the general training provided at 
DLIFLC. We also ask students for their opinions and attitudes at various time 
through questionnaires and interviews. All this is again part of our 
continuing effort to provide the best language training possible for DLI 
students. 

The DLI staffs in cooperation with the Army Soldier Support Center and the 
Army Research Institute, hasa plan to try but two kinds of methods of 
teaching Russian with your class. We plan to try these but with fbur of the 
sections of this class. With_10_ students per sections^ we will use methbd "A" 
in two sections for a total of 20 students. With twb other sections br a 
total of 20 students, we will use method "B". This study will btHy last 
daring the first term of the Basic Course or approximately 15 weeks. At that 
time a Proficiency Advancement Test (PAT Ij is given to all students anyway. 
Ail students are expected to pass PAT I bef ore moving oh to Term II. 

Both groups will cover essentially the same amount of material that is 
covered by all students in Term I of the Basic Course^ except that there will 
be differences in what goes oh in the classroom. The learning and teaching in 
two of these sections using method "A" will be somewhat different than the 
learning and teaching in the thUi sections normally used in DLI classes. Two 
other sections will follow the normal DLI pattern of teaching and learning 
(Method "B"5 except that records kept on all four groups will be analyzed and 
compared in much more detail. Alsb^ there may be somewhat more attention 
given to observations of the classroom activities and to the progress 
individual students are making. There will not be any extra work of any kind, 
so the same amount of effort will be expected anyway for students in all 
sections. 

The Army is fully behind this study and expects service members to apply 
themselves by learning Russian as their basic duty regardless of which section 
they may be assigned to. Normal assignments to sections in DLI Basic Courses 
is controlled by the language department chairpersons and their supervisors. 
We do not make up sections artificially by putting all the people with high 
scores on tests such as DLAB and/or ASVAB in one section and all the low 
scorers in another. Rather we normally seek balanced grouping in all 
sections, so there is no special advantage in being in one section over being 
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In another one. For example, we prefer to have the same ratio of males and 
females in all sections. 

Once you are assigned t5 8ne of these four sections, you are expected to 
remain in it just as all students assigned to any sections are expected to 
remain with their section until such time as there may be an administrative 
reason for recombining sections. This would not normally happen until after 
15 weeks anjrway. We assume that you will put your best efforts into learning 
Russian regard-less of which section you are assigned to and without favor to 
one supervisor over another or one set of instructors over another. 

You should be aware that proponents for method "A" claim that people learn 

much faster when it is used. This is not necessarily unusual, since 
proponents for innovative methods almost always m^e such claims. DLI's 
purpose ±s to give a fair chance for method "A" to be fully evaluated in 
comparison with DLI's standard method o€ teachings Incidentally, DLI's 
standard method also has many innovativ: features, especially since the entire 
course, including materials and tests, is quite new and is still subject to 
some revision. So just consider that the normal mode for all DLI sections is 
not cast in concrete but rather is still open to change as we never stop 
seeking to improve all of our training materials and methods so that our 
students can learn more effectively and efficiently. 

If any of ybU think you will feel uncomfortable by being in one of the 
four sections destined to be given more evaluation, please say so now. Gnce 
we make assignments to sections we do not expect to make changes. 

Are there any questions? 
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APPENDIX J 



DESCRIPTIONS AND RATINGS 

Table J-l. Ratings for the Speaking Portion of the PAT, and for the 
(Face-to-Face) Oral Interviewis 
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INTERAGENCY LANGUAGE ROUNDTABLE 
LANGUAGE SKILL LEVEL DESCRiPtiONS 

FOR SPEAKING (PAT) AND ORAL INTERVIEW RATINGS 



Preface 

The following proficiency levei descriptions 
characterize spoken iangpage pse. Each oj the six 
•'base leyelf; {coded 00. 10. 20. 30. 40. and 50) 
implies controj of an^jjrevipus "base leversV* 
functions and accuracy. The '"pLus leyer" 
desighatibh (coded 06. 16. 26. etc.) will be 
assigned when prof iciency substantially exceeds 
one base skill level and does hot fully meet the 
criteria for the next "base level." The "J5lus level" 
descrlpjipns are therefore supplementary to tne 
"base level" descriptjons. 

A skill lev^l Is assigned to a person through an 
authprlzed language examinalion: Examiners 
assign a level on ayariety of perfbrmance r rite' 
exemplijied in the descrjptjve statemen 
Therefore, the examples given here illustrate, i A 
do noj exhaustiyejy describe, either the skiJis ■^ 
person may possess or situations in which he/^' •'^ 
may function effectively. 

Statements describing accuracy refer to t^rplcal 
stages in th^ deyelqpmfht of cbinpetehce in the 
most commonly taught languages in formal 
training programs. In other languages^ emerging 
competencej^rrallels thes^ 
but often with different details. 

Unless otherwise specified, the term "native 
speaker • refers to native speakers bf a standard 
dialect, 

•^Well-educated. " in the context of these 
proficiency desc/iptidhs. dbes hot niacessarlly 
imply forma! higher edujjatiqri. However, m 
cultures where formal higher educatjort is 
common, the language-use abilities of persons 
who have had such educajipn is considered the 
standard- That Is^ such a person meets 
contemporary expectations for the formal, 
careful style of the language, is well as a range of 
less formal varieties of the language. 

Level d (No Proficiency) 

Uhible to function iri the spoken laniuade. 
Oral prqducJlqh Js Jimited tbbccasJonal Isolated 
words. Has essentially nq cd_mmunicative abiljty: 
[Has been coded S-b In some nonautomated 
applications.) [Computer Code 00]| 

Level 0+ (Memorized Proficiency) 

Ablii to f atJify immeaiAte needs using 
rahearsed ansrsncM. Shbws little reaJ adtbhomy 
of expression, flexibliity, or spontaneity. Can ask 
questions or make statements with reasonable 



accuracy only with memorized utterances or 
formulae. Atte»^ at creating speech are usually 
unsuccessful. 

Examples: The jndlyiduars vocabulary is 
usuaMy limited to areas of immediate survival 
needs.__Mdst uXterahces are telegraphic; that is. 
functors (linking words, markers, and the like) are 
bmltted. confused, or distortetf. An individual can 
usually differentiate most significant sduhds 
when produced in isolation, but. when combined 
in words ^r groups of wqrds. errbrs may be 
frequent. Even with repetition, communication is 
severely limited even with people used to dealing 
wjth f<:?reigners. Stress, intdhatibh, tone, etc are 
us lally quite faulty (Has been eoded S-0+ in 
^ome nonautomated applications [Computer 
Code 06] 

Level 1 (Elementjury Proficiency) 

# 

Able to ntlsfy mlnimufYi courtidiy requirements 
end muljitain vory simple facr-td^faco 
conversations on familiar topics. A native speaker 
must often yse_ slowed speech, repetition, 
paraphrase, or a combirnatidh bf these to be 
understood by this individual. Similarly, the 
native speaker must strain and employ reai-wqrid 
k n owl edge to under s t a h d e v en s i m p j e 
statements/questions from this^ individual. This 
speaker has a functional, butlimitedproficiency 
Misunderstand ihgs are frequent, but the 
mdividual is able tq ask for help arid to verify 
cqmprehensibri of native speech in face-to-face 
interactipri. The individual is unable to prqduci 
continuous discourse except with rehearsed 
material. 

Examples: Structural^ accuracy is likely to be 
randbm br severely limited: Time concepts are 
vague^ yqcabulary is inaccurate, and Its range is 
very narrow: The individual bfteri speaks with 
great difficulty: By repeatin^^ such speakers can 
make themselves understood to najlye speakers 
who are in rejular cqritact with foreigners but 
there is little precisLon in the information 
conveyed. Needs, experience, or traimrig may 
vary greatly frbrti iridlvidual to individual; for 
example, speakers at this level may have 
encountered quite different vocabulary areas 
However,_ the individual can typicalOy satisfy 
predictable, simple, persbrial arid accommodation 
needsi can generally meet courtesy. Introduction 
a n d I d_e n t i f i ca ti o n r e q u i r e nri en t s ;_e x c h a ri g ^ 
greetings. eMcit_ arid provide, for example 
predictable and skeletal biographical 
Information. He/she might give information about 
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busihiss hbufs,_exj3laih_rputi^^^^^ jh a 

Limited way, and state^in a simple manner what 
actions will betaken^ He/she is able to formulate 
some questibhs even in languages with 
com pi i cated g u estion construct ons. AJrn ost 
every utterance may be characterized by 
structural errors and errors in basic gramrhatical 
tflitiPM. ypcabu lary is exj rernely Jim ijed arid 
characteristically does not include modifiers. 
Pronunciatibh, stress, and ihtonation are 
girierally poor, ofteri heavily irifluericed by 
a n q t h e r I an g u age. Use c f st r u c t u r e a Q d 
vocabulary is highly imprecise (Has been coded 
S-1 in some hbhautbmated applications.) 
f Cbmputer Code 10] 

Level 1+ (Elementary Proficiency, Plus) 

Can initiate and maintain predictable face-to- 
face cbriversatibht and satisfy limited social 
dimindi. He/she may^ however, have littLe 
understanding of the social conventions of 
conversation. The interibcutbr is geheralJy 
required to stram arid employ _real-_wpf Id 
knowledge to understand even some simple 
speech, the speaker at this level m^ 
may have to change subjects due to 
language resources. Range and control of the 
[anguage are limited. Speech largely consists of a 
serJes bf shbrt. discrete utterances. 
_ _EximpLos:_The irtdj is abLe to satLsfy mosj 
travel and accommodation needs and a limited 
range of social demands beyond exchange of 
skeietal bibgraphjc mfbrmatibn. Speaking ability 
may extend beyond [mmediate survival needs. 
Accuracy in basic grarnLrhatical relations Is 
eyidinti although ribt cbrisi^sterit May exhibit the 
mo re^cqm^o n forms of verb tenses^f or exa^ pie, 
but may mai^^ frequent errors In formation and 
selection. While sbme structures are established, 
6J'!?>''S_pccur in more c]^^^ 
individual typically cannot sustain coherent 
structures in longer Utterances br unfamiliar 
situations. Ability to describe amJ ^ive j5f ecise 
Infoj'mation is Jirnlted: Person, space, and time 
references are often used iricbrrectly. 
f^ronuncjation js urider^ to naliyes used 

to dealing with foreigners^ Can combine most 
significant sounds with reasonable cbrhpreheh- 
8ibilijy,_ but has_ difficuity in producing certain 
sounds in certain positions or in certain 
cbrhbihatJbns. Speech will usually be labored. 
FriQUPritly has tb repeat utterarices tp be 
undersjood by the general public. (Has been 
coded S-l-f- in some aonautomauSd applications.) 
[Computer Cbde 16] 

Level 2 (Limited Working Proficiency) 

Abt% td iitjffy routini ioclil djfmM ihd 
iimitsd work rsqairtmtnts. Can handle routine 
work-related interactions that are llrnited in 
scope. In more complex and sophisticated work* 



related tasks, Languag^e usage generally disturbs 
the native speaker. Can handle with confidence 
but not with facility, rhbst horrhal. high-frequehc) 
s p c i a I c b rt y is r s_a t jb rt a I s i t u a t i o n s m c [u d i n | 
extensive, but casual conversations about curren 
events, as well as vi^ork. famiJy. aric 
autobiographical iriforrhatibri. The iridiyidual car 
get the alst of most every^day conversations 
has some difficulty understanding native 
speakers in situatibhs that require specialized bi 
sop h i s t i c at ed knowledge^ The in d iyi dual's 
utterances are minimally cohesive: Linguistic 
structure Is usually ^bt v<;ry elabbrate and nb 
thbrbughly cbritrblled; errors are frequerit 
Vocabulary use is appropriate for high-frequenc> 
utterances^, but unusudi or imprecise elsewhere 
Examples: While these interactibhs will var> 
widely from individual tojndlvldual, the Indlvidua 
can typically ask and answer predictable 
questions iri the workplace and give 
straightforward ^.^ subordinates 
Additionally, the 'ivHividual can participate in 
persbnal ahd accbmmbdatibh*type interactions 
vvjth iiabpratipri arid facility;, that is, can give and 
understand complicated, detailed, and extensive 
directions and make hoh-routihe changes in 
travel arid accpnlrnbdatiori^ar^^^^ 
structures and basic grammatical relations are 
typically cohtrbried; however, there are areas 6i 
weakness. In the cbmmbrily taught lariauages, 
these m ay be^ simple mark i ngs such as p I ura Is . 
articles, linking words, and negatives or more 
cbmplex structures such as tehse/asjDect usage, 
case morphoiog^, passive cqn^^^ word 
order, and embedding: (Has been co&i^ S*2 in 
some nonautomated applications.) [C^^mputer 
Cbde 20] 

Level 2+ (Limited Working Proficiency, Plus) 

Able to satisfy most work requlrernenjs with 
langaage usage that Is often, bat not always, 
acceptable and effedtive. The individual shbws 
corisid^e rab I e ability to communicale effec^^ 
on topics relating to particular interests and 
special fields bf competence. Often shows a high 
degree 01 fluency and^^^ 

under tension or pressure, the abihty to use the 
Ijahgu&ge effectJvely rnay deteriorate. 
Ppmprehensipn of^rio^^ Jriative speech is 
typically nearly complete: The individual may 
miss cultural and local references and rnay 
require a native speaker, to adjust to his/her 
[imitations in some ways. Native speakers often 
perceive the Jhdlviduars speech to contain 
awkward br inaccurate phrasing bf ideas, 
rnisjaken t[me, space, and person references^ or 
to be In some way inappropriate, if not strictly 
ihcbrrect. _ 

Examj^lesL TypicaMy thejridivjduaj_ can 
participate In most social, formal, and informal 
interactions; but limitations either in range of 
contexts, types of tasks, or level of accuracy 



Kinder effectiver^^^ individual may be Ijj at 

ease with the use of the Janguage either in social 
ihteractibh or in speaking at leng^th in 
protessional contexts. He^ is generally strong 
in either structural precision or vocabulary 
hot in both. Weakness or unevenness In one of the 
foregoing^ or in prbnuhciatJdh, bccaslonaiiy 
results in miscommunLcatiOh.No^rmally controls^ 
but cahnot always easily produce general 
y<^abujary._ Oiscourse is often ihcohesive. (Has 
been coded in sOTne r^bnautomated 

applications.) [Computer Code 26] 

Level 3 (General Prbfessibnal Proficiency) 

Able to 4r/^ak the Umguage with jiuffi^^^ 
ttni^;! :i;uracy and vocabulary to participate 
•ffec^Jvctv in most formal and ihfbrmaj 
ctr»^f^r%mWoh9 on pracflcal, sociai, and 
pro; ^ iai tppioi. Nevertheless, the individual's 
limitations generally restrict the prbfessibnal 
cphtexts of language use to matters of shared 
knqwledjie and/pr ihternatibnaJ convention. 
Discourse is cohesive. Th. individual uses th^ 
language acceptably, but w3th some noticeable 
imj^rfectibns; yet, errbrt^ <'':!oa?? ; never interfere 
with underatandjng aiTtf »crely tl>%turb the mrye 
speaker. The individual can et^*c*^V3iy cbmbme 
strtictyre and vbcabuiaiy to crr.vey his/her 
meaning accurately. The individual speaks 
readijy and fills pauses suitably. In face-tb-face 
cbhyersatibn with natives speaking the standard 
diaject at a normar rate of speech^ cbrhprehens ion 
is qujte complete. Alt^ou^h cuitural references, 
proverbs, and the ImpMcations of nuances and 
Wipm rtiay hbt be fully uhderstbbd,Jhe individuai 
can easily repah the conversatibh. Prbhuhciatibn 
may bfe obviously foyaign. Individual sounds are 
accurate; but stress, intonation , and pitch control 

maybefauLty. 

Examples: Can typically discuss particular 
interests and special fiejds of competence with 
reasonable ease. Can use the language as part of 
normaJ professional duties such as ahswerihg 
bbjectibhs^ clarifying points, justiiyinadeclsibhs, 
understanding the essence of challenges, stating 
and defending policy,_ conducting meetings, 
delivering briefings, or other extended and 
ejabprate ihfbrmative monologues. Can reliably 
elicit infqmiimon and inf^^ bpinlbh from 
native speakers. Structural inaccuracy is rarely 
the major cause of jmlsunderstanding. Use of 
structural deLvlce^^ is flexibJe and elaborate. 
Without searching for vvprds or phrases,^ the 
Individual uses the language clearly and relatively 
naturally to ejabprate cbhcepts freely and make 
ideas easily understandable to native speakers. 
Errbrs occur ^n low-frequency and hijhiy 
/ complex structures. (Has been coded S-3 in some 
nonautomated applications.) [Computer Code 
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Level 3+ (General Prbfessibnal Proficiency, 
Plus) 

is oftM i&ie to me the tenguage to iitlsfy 
profesfjoiiai needs In a wide range of 
sophlsticaje^d and jlemindihg tmiki. 

Exainples: Despite obvious strengths, may 
exhibit some Hesitancy, uncertainty, effort, of 
errors which Hmit the range of laogoage-use tasks 
that can be reliably performed. Typically there Is 
particular strenjjth ini fluency and one or more, 
but not an, of the follpwirig: breadth of lexicon, 
Including low- and medium-freqUehcv items, 
especially socio-lingujstic/cultural references 
and nuances b^close synonyms; structural 
precjaion, with sophisticated features that ire 
readily, accurately, and appropriately cbntrblled 
(such as cbmplex mbdificatibh and embedding in 
Indo-European languajLes); discburse 
cbmpetehce in a wide range of contexts and 
tasks^ cmen matching a native speaker^s strategic 
and organizational abilities and expectations. 
Pccasibhal patterned errors occur ih low 
freguencx arid highly-complex structures. (Has 
been coded S-3+ in some abnautbmated 
applicatibhs.) [Computer Code 36] 

Level 4 (Advanced Professional Proficiency) 

Able 16 use ihe Jangaage f jyf ntly and 
iccurjLtely ph all levels normally pertinent to 
professional needs. The individual's language 
usage and abilrty to function are fully successful. 
Organizes discburse welh using appropriate 
rhetorical sjseeclLdev^^ native cultural 
references, and understanding. Language ability 
only rarely hinders him/her in performing any 
task reguiring^lahgUageLyet, the Individual would 
seldom be perceived as a native. Speaks 
effortlessly and smoothly and is able to usf the 
language wUh a high degree of effectiveness, 
reliability, and precision for all representational 
purpbses within the range of personal and 
professiphal experience and scope of 
responsibilities^ Qih serve as an Informal 
interpreter in a range of unpredictable 
circumstances, Cah^perform extensive^ 
sophistlcated lahguage tasks, encbmpassing 
most matters of interest to well^ducated native 
speakers, including tasks which do not bear 
dlrectl£pn a prbfessibnal specialty. 

Examples: Can discuss Jn detail cbhcepts 
which are fuhdamentajly different from those of 
the taset culture and make those concepts clear 
and accessible to the native speaker. Sirnllarly, 
the individual can understand the details and 
ramlflcatiphs bf cbhcepts that are culturally or 
cojiceptually diffejent from his/her own. Can set 
the tone of Interpersonal offlciaLjemlHOff 
arid hbh-prbfesslohal verbal exchangej with a 
representatlye range speakera (Vn a 

range of varied audiences, purposes, tasks, and 



settingsJ^Can play an effective role among native 
speakers in such contexts as conferences, 
lectures, and debates oh matters of disagreement. 
Can advocate a pojrtjor^^^ length, both formally 
and in chance encounters, using sophisticated 
verbal strategies. Understands and reliably 
Pfpduc^^_ shafts o 

Can understand native speakers of the standard 
and blher major dialects in essentiallyany face-to- 
face, Ihteractipn. [Has b^ cpded S^ in some 
nonautomated applications.) [Compater Code 40] 

Level 4+ (Advanced Professional Proficiency, 
Plus) 

Speakir g proficiency Is negiilariy iuperibr In all 
retpeett, usually equlvaient to Jhat of a weH- 
educated, h[ghiy artlcuiate native speaker. 

Language ability does hot impede the 
performance of any language-use task. However, 
the individual would not necessarily be perceived 

as cuJtufaHy hativa 

Examples: The individual organizes discourse 
well, employing functional rhetorical speech 
d ey i_c e_s^ r[a t l_v e c_u 1 1 u r a I r e f f r e h ce s a h d 
understanding. Effectively applies a native 
speaker's sbciai and circumstan^tial knowledge. 
However, cannot sustain that performance under 



all circumstances. While the individual has a w|de 
range and control of structure^ an occasionai non- 
native slip rria^ bccur. The individual has a 
sophisticated cpntroi of vcjcabuiary andjjhr^^^ 
that Is rarely imprecise, yet there are occasional 
weaknesses in idioms, collbquiailisms, 
Pronuru:jation, cultu^^ 

an occasional failure to interact in a totally native 
manner. (Has been codid S-4+ in some 
hdnautdm&lsd applications.) [Computer Code 46] 

Level 5 (Functionally Native Proficiency) 

Speaking projlclency Is functionally equlvalenj 
to that of a highly arllculafe welJ-educated native 
speakfr and reflects the cultural standards of the 
country where the language Is natively spoken. 

The individual uses the language with complete 
Mxibility|mcJir[^ 

is faiiy accepted by well-educated native speakers 
In all of its features, including breadth of 
y pc aj) u l a r y and i ^iLPni , c p J I o ctu La I Ls m s , and 
pertinent cultural references. Pronunciation is 
typically consistent vy^ith that of vy^ell-educated 
native speakers of a npn-stigniatized dialect. (Has 
been coded S-5 in some nonautomated 
applications.) [Computer Code 50] 
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